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SUMMARY 


This  paper  documents  the  effort  to  develop  items  for  the  next  generation  of 
Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  Forms  20,  21,  and  22.  The 
ASVAB  is  a  multiple  aptitude  test  battery  that  is  used  by  each  of  the  Armed 
Services  to  select  enlisted  personnel  and  to  classify  the  personnel  into  occupations. 
New  forms  of  the  ASVAB  are  required  to  reduce  test  compromise  and  to  update 
content.  New  forms  must  be  content  and  statistically  parallel  to  one  another  and  to 
a  reference  form,  Form  8a.  Over  2,500  experimental  items  were  written  and 
administered  along  with  the  ASVAB  Form  8a  items.  Experimental  and  Form  8a 
item  statistics  were  calculated  to  determine  if  sufficient  numbers  of  experimental 
items  matched  the  Form  8a  items.  It  was  concluded  that  adequate  numbers  of 
experimental  items  matched  the  Form  8a  items  and  that  six  new  parallel  forms  of 
the  ASVAB  could  be  const  meted  in  a  subsequent  effort. 


PREFACE 


This  paper  documents  the  efforts  conducted  in  support  of  Prototype 
Development  and  Validation  of  Selection  and  Classification  Instruments  (Contract 
F41689-87-D-0012).  This  research  and  development  (R  &  D)  effort  was  conducted 
under  the  Development  of  the  Armed  Services  Vocational  Aptitude  Battery  Forms  20, 
21,  and  22  (Items  and  Item  Bank)  by  Operational  Technologies,  Inc.  (OPTECH), 
San  Antonio,  Texas. 

We  would  like  to  express  our  sincere  appreciation  to  Air  Force  Human 
Resources  Laboratory  scientists  Dr.  Malcolm  James  Ree,  for  his  technical  insights, 
and  2Lt  Stephen  D.  Armstrong  for  his  assistance  in  publishing  the  final  copy.  We 
also  thank  OPTECH  personnel  Juanita  Nakasone  and  Laurie  Shore  for  their  data 
entry  services.  The  contributions  of  these  individuals  were  essential  to  this 
project. 
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ARMED  SERVICES  VOCATIONAL  APTITUDE  BATTERY  (ASVAB) 
FORMS  20,  21,  AND  22:  ITEM  DEVELOPMENT 


I.  INTRODUCTION 

The  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  is  the  multiple 
aptitude  battery  used  by  all  the  armed  services  for  selection  and  classification  of 
enlisted  personnel.  The  ASVAB  consists  of  10  multiple  choice  subtests,  8  power 
and  2  speeded.  These  10  subtests,  their  content,  the  number  of  items,  and  time 
limits  arc  shown  in  their  order  of  administration  in  Table  1. 

Tour  subtests  are  used  to  compute  the  Armed  Forces  Qualification  'Test 
(APQT)  score.  This  AFOT  composite  is  calculated  from  the  Arithmetic  Reasoning 
(AR),  Word  Knowledge  (WK),  Paragraph  Comprehension  (PC),  and  Mathematics 
Knowledge  (MK)  standard  scores  and  is  used  to  determine  the  qualification  of  an 
applicant  into  any  branch  of  the  service.  All  the  subtests  are  used  in  various 
Service-specific  composites  for  determining  an  individual’s  training  and  job 
assignment. 

Some  versions  of  the  ASVAB  are  administered  in  high  schools  for  future 
recruiting  and  for  counseling  purposes.  However,  operational  ASVAB  versions  are 
administered  to  applicants  at  military  entrance  processing  stations  (MEPS),  mobile 
examining  team  (MET)  sites,  and  Office  of  Personnel  Management  (OPM)  sites. 

The  operational  ASVAB  versions  periodically  require  the  development  of  new 
test  batteries  to  provide  optimum  test  effectiveness  in  the  allocation  of  enlisted 
resources  and  to  deter  test  compromise.  Effective  job  classification  results  in  cost 
reductions  due  to  higher  productivity,  higher  reenlistment  rates  and  reduced  training 
school  attrition. 

The  development  of  new  operational  ASVAB  forms  is  accomplished  in  four 
major  phases: 

Phase  I:  Items  are  written,  edited,  checked  for  technical  accuracy,  and 
reviewed  for  population  subgroup  sensitivity.  Booklets  are  developed  and 
administered  to  Air  Force  recruits  to  obtain  preliminary  item  statistics. 

Phase  II:  Content  and  statistically  suitable  items  are  selected  from  Phase  I,. 
Overlcngth  booklets  are  developed  and  booklets  are  administered  to  al  1  - 
Service  recruits  to  obtain  item  statistics  based  on  a  sample  with  a  broader 
range  of  ability. 

Phase  III:  Content  and  statistically  suitable  items  are  selected  from  Phase  II. 
Operational  length  booklets  are  developed  and  trial  equatings  of  the  new 
operational  tests  with  the  reference  test  are  performed. 

Phase  IV:  Operational  length  booklets  created  in  Phase  III  are  administered  to 
all-Service  applicants  to  obtain  item  and  subtest  statistics  based  on  a  sample 
with  a  broader  range  of  ability.  Equatings  of  these  operational  booklets  with 
the  reference  booklet  are  performed. 

I  he  locus  of  this  paper  is  the  Phase  I  stage  of  developing  the  new  items  for 
ASVAB  Forms  20  to  22.  T  his  study  encompasses  three  purposes: 
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1.  the  development  of  an  adequate  number  of  items  to  generate  six  ASVAB 
forms  that  will  ultimately  be  content  and  statistically  parallel  to  each  other 
and  to  the  reference  ASVAB  Form  8a  and, 

2.  the  configuring  of  experimental  and  8a  booklets  for  item  testing,  and 

3.  the  administration  of  these  booklets  to  obtain  preliminary  item  statistics 
that  will  be  used  in  a  later  effort  to  develop  overlength  forms. 


Table  1 ■  ASVAB  Subtest  Descriptions 


Subtesta 

Content 

1  toms 

Time 

General 

Sc i ence  (  GS  ) 

Knowledge  of  or  about 
physical,  chemical, 
and  biological 
proper  t i es 

25 

11 

Ar i thraet 1 c 

Reasoning  ( AR ) 

Reasoning  required  to 
perfoim  arithmetic 
processes 

20 

3  fa 

Word  Knowledge  (WK) 

The  meanings  of 
selected  words 

35 

1  1 

Paragraph 

Comprehension  (PC) 

Understanding  of 
written  material  from 
brief  paragraphs 

15 

13 

Numerical  , 

Operations  (N0)° 

Knowledge  of  simple 
addition,  subtraction, 
multiplication,  and 
division 

50 

3 

Coding  Speed  (CS)b 

Ability  to  identify  and 
match  similar  sets  of 
numbers  with  words 

84 

7 

Auto  and  Shop 
Information  (AS) 

Knowledge  of  and 
familiarity  with  tools 
and  shop  practices,  and 
knowledge  of  and 
familiarity  with 
mat ntenance ,  structure, 
and  repair  of  automobiles 

25 

1 

Mathematics 

Knowledge  ( MK ) 

Application  of  learned 
mathematic  principles 

2  5 

24 

Mechanical 
Comprehension  (NC) 

Understanding  and 
application  of  various 
mechanical  principles 

25 

19 

Electronics 
Information  ( KI ) 

Identification  or 
application  of  simple 
electric;'.]  or 
electronies  knowledge 

20 

9 

f  Subl  c  si  s  arc  presented  in  ordet  of  administration. 
’  Speeded  suhtest 


II.  METHOD 


Determination  of  the  Number  of  Items  within  a  Subtest 


In  determining  the  number  of  Phase  I  items  to  be  generated,  the  convention 
exists  that  six  unique  parallel  versions  of  the  AFQT  portion  of  the  battery  will  be 
developed  and  three  unique  parallel  versions  of  the  non-AFQT  portion  of  the  battery 
will  be  developed.  Thus,  determining  the  number  of  items  to  be  developed  is 
contingent  on  the  length  of  the  subtest  in  operational  format,  whether  it’s  an  AFQT 
or  non-AF'QT  subtest  and  the  number  of  forms  to  be  generated.  Given  these 
considerations,  the  number  of  items  is  then  tripled  to  ensure  sufficient  numbers  of 
items  throughout  Phase  II  and  III  analyses.  The  speeded  subtest  items  were  not 
generated  in  this  study  as  they  are  typically  produced  during  Phase  II.  The  total 
number  of  items  needed  for  Phase  I  of  ASVAB  Forms  20  to  22  are  presented  in 
Table  2. 

Further,  this  effort  was  able  to  use  power  subtest  items  contained  in 
previous  operational  booklets  (see  Appendix  A).  Approximately  20%  of  the  total 
number  of  items  for  the  item  development  stage  of  ASVAB  Forms  20  to  22  were 
previously  used  items.  Of  these  items,  half  were  revised  and  regarded  as  new 
items  and  half  were  used  verbatim. 

Once  the  number  of  items  for  Phase  I  was  determined,  item  writing  for  the 
experimental  booklets  began. 


Table  2 . 


Sub tral  a  .me 


Phase  I  Item  Requirements*1 


Number  of 

Number  Items  required  Tote.',  number 

of  forms _ per  form” _ of  items 


AFQT  sub  test  s 


Word  Knowledge  (  WK ) 

6 

35 

X 

3 

630 

Paragraph  Completions  1  on  (PC) 

6 

15 

X 

3 

270 

Arithmetic  Reasoning  (AR) 

6 

30 

X 

3 

540 

Mathematics  Knowledge  ( MK ) 

6 

25 

X 

3 

450 

Non-AFQT  subtests 

Electronics  Information  (El) 

3 

20 

X 

3 

180 

Mechanical  Comprehcns 1  on 

3 

25 

X 

3 

225 

General  Science  ( GS ) 

3 

25 

X 

3 

225 

Auto  ana  Shop  Information  (AS) 

3 

25 

X 

3 

225 

Total 

2,745 

*  Speeded  subfesls  .VO  and  C'. S  ate  not  developed  in  Phase  I. 

Three  times  (x  3)  as  man  v  items  ns  required  for  the  operational 
length  sublesl  were  written. 
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Hem  Writing 

With  the  total  number  of  items  that  needed  to  be  written  determined,  the 
following  guidance  was  given  to  item  writers.  First,  item  writers  were  given  a 
taxonomy  for  each  subtest  indicating  the  percentage  of  items  that  needed  to  be 
covered  in  particular  content  areas  (see  Appendix  B).  Given  these  percentages  and 
the  predetermined  total  number  of  items  for  each  subtest,  the  number  of  items  that 
needed  to  be  written  per  content  category  were  determined.  The  use  of  "all  of  the 
above"  and  "none  of  the  above"  as  item  alternatives  was  prohibited.  Item  content 
taken  verbatim  from  copyrighted  materials  was  not  acceptable.  Item  writers  could 
not  use  items  present  in  current  operational  military  tests  or  in  books  used  for  test 
preparation.  For  the  PC  subtest,  at  least  2  to  5  items  needed  to  be  written  for 
each  paragraph. 

In  addition  to  the  new  items,  revised  and  unrevised  previously  used  items 
were  included  in  each  subtest’s  item  pool  for  a  percentage  not  to  exceed  20%  for 
revised  items  and  20%  for  unrevised  items.  For  many  of  the  subtests  there  were 
not  enough  old  items  available  for  reuse  in  the  development  of  forms  to  meet  these 
percentages.  This  meant  that  over  60%  of  the  item  pool  contained  new  items 
written  in  order  to  approximate  the  total  number  of  items  needed  for  each 
taxonomic  category. 

Booklet  Configurations 

After  all  power  subtest  items  had  been  written  according  to  the  criteria, 
experimental  forms  and  the  reference  Form  8a  booklets  were  configured. 

Experimental  booklet  configurations  in  Phase  I  were  based  on  the  number  of 
items  to  be  included  in  each  overlength  subtest  which,  in  turn,  was  contingent  on  the 
total  number  of  items  needed  for  each  subtest.  Complete  ASVAB  booklets,  in 
ovcrlcngth  format,  were  not  constructed  because  the  testing  time  was  limited  to 
3  1/2  hours.  Partial  booklets  consisting  of  a  subset  of  ASVAB  subtests  were 

configured  with  equal  numbers  of  items  in  the  8a  and  experimental  subtests. 

Certain  experimental  suotests  were  grouped  together  to  form  a  partial  booklet 
and  were  assigned  an  Air  Force  Human  Resources  Laboratory  (AFHRL)  booklet 
number  (see  Appendix  C).  ASVAB  booklet  MO88057  contained  General  Science 
(GS),  Mechanical  Comprehension  (MC),  Auto  and  Shop  Information  (AS),  and 
Electronics  Information  (El)  8a  subtests.  ASVAB  booklets  M 088058  through 
MO88063  contained  GS,  MC,  AS,  and  El  experimental  subtests.  MO88064  contained 
GS,  MC,  AS  but  did  not  contain  the  El  subtest  because  fewer  El  items  were 
required.  ASVAB  booklet  MO88065  contained  Word  Knowledge  (WK),  Paragraph 
Comprehension  (PC),  and  Mathematics  Knowledge  (MK)  8a  subtests  while  ASVAB 
booklets  MO 88066  through  MO88078  contained  WK,  PC,  and  MK  experimental 
subtests.  Booklet  M  0880/9  contained  only  WK  items  since  fewer  PC  and  MK  items 
were  required.  For  the  remaining  subtest,  Arithmetic  Reasoning  (AR),  no 
ovcrlcngth  booklets  were  made  so  that  examinees  would  not  become  fatigued  during 
the  testing  session.  The  AR  subtests  (M088CP4,  8a;  MO88095  through  M088112, 
experimental)  were  single-subtcst  booklets  wnich  resulted  in  the  need  for  more 
testing  sessions  to  administer  the  total  number  of  items  required. 

Administration  Plan 

An  administration  plan  and  administration  instructions  were  created.  The  goal 
of  administration  was  to  produce  randomly  equivalent  groups  for  all  booklets. 
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Phase  I  testing  of  experimental  and  8a  partial  booklets  was  conducted  at  Lackland 
Air  Force  Base  (AFB),  Texas,  between  April  1988  and  January  1989.  The  partial 
booklets  were  administered  with  their  own  answer  forms  according  to  the  following 
plan  with  a  typical  session  testing  approximately  150  subjects: 

•  The  experimental  AR  booklets  containing  30  experimental  items  were 
randomly  distributed  with  approximately  every  8th  person  receiving  the  8a  AR 
booklet,  also  containing  30  items.  This  resulted  in  at  least  540  subjects  being 
administered  each  of  the  14  AR  experimental  booklets  and  at  least  1,080 
subjects  being  administered  the  AR  8a  items  across  all  testing  sessions. 

•  The  experimental  VVK  series,  containing  44  WK  items,  21  PC  items  and  34 
MK  items  were  randomly  distributed  with  approximately  every  6th  person 
receiving  the  8a  counterpart.  This  resulted  in  at  least  560  subjects  being 
administered  each  WK  series  booklet  and  at  least  1,120  subjects  being 
administered  the  WK  8a  series  across  all  testing  sessions. 

•  The  experimental  GS  series,  containing  31  GS  items,  31  AS  items,  31  MC 
items  and  20  El  items  were  randomly  distributed  with  approximately  every  4th 
person  receiving  the  GS  8a  counterpart.  This  resulted  in  at  least  525 
subjects  being  administered  each  GS  experimental  booklet  series  and  at  least 
1,050  subjects  being  administered  the  GS  8a  series  across  all  testing 
sessions. 

If  all  three  booklet  series  were  administered,  AR  with  a  separate  answer 
form  was  administered  first.  The  AR  booklets  and  answer  forms  were  collected 
and  the  WK  series  with  another  answer  form  was  distributed.  The  WK  booklets 
and  answer  forms  were  collected  and  the  GS  series  with  another  answer  form  was 
distributed.  The  GS  booklets  and  answer  forms  were  collected  and  the  subjects 
dismissed.  The  total  testing  time  for  this  type  of  session  was  approximately  2  3/4 
hours. 


If  only  2  booklet  series  were  administered,  the  above  procedure  was  followed 
with  the  administration  order  being  the  AR  booklet  then  the  WK  series,  or  the  AR 
booklet  then  the  GS  scries.  The  total  testing  time  was  about  1  3/4  hour. 

Instructions  for  administering  each  series  of  partial  booklets  were  developed 
to  insure  standard  administration  for  all  testing  sessions.  These  instructions  were 
extracted  from  the  standard  ASVAB  administration  manual  (Department  of  Defense, 
19S3)  and  adapted  to  accommodate  the  partial  booklet  testing  format. 

Subjects 

Item  responses  were  collected  from  samples  of  approximately  500  to  800  for 
each  experimental  sublest  and  approximately  1,000  for  each  reference  subtest. 
These  samples  consisted  of  male  and  female  Air  Force  recruits  undergoing  basic 
military  training  at  Lackland  AFB,  Texas.  All  subjects’  answer  forms  were 
scanned,  and  for  ease  of  subsequent  data  processing  individual  experimental  and  8a 
booklet  session  files  were  built.  Since  fewer  than  100  subjects  w'ere  administered 
the  8a  items  in  a  given  session,  the  8a  item  responses  were  accumulated  across 
testing  sessions  so  that  item  statistics  could  be  estimated  using  a  large  sample. 
J  he  number  of  Air  Force  recruits  present  at  each  testing  session  is  shown  in 
Table  3. 
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Table  3.  Total  Number  of  Subjects  Per  Session 


Session  # 


Booklet  number  #  of  subjects 


1  M088094 ( AR-8a )  60 

M088095  AR-X)  559 
M088065 (  J”-8A )  80 
M08S06  K-X)  539 

2  M0880'4V AR-8A)  65 

M088096( AR-X  )  592 
M088065 ( WK-8A )  84 
M088067( WK-X )  578 

3  M088094 ( AR  8A )  65 

M088097( AR-X  )  570 
M08806  5 ( WK-8A  )  82 
M088068( WK-X )  556 

4  M08S094  ( AR-8A )  65 

M088098  AR-X)  586 
M088065 [ WK- 8A )  82 
M088069( WK-X  )  570 

5  M088C94 ( AR-8A )  66 

M088099 ( AR-X  )  629 
M088057( GS-8A )  179 
M008058( GS-X  )  525 

6  M088094 ( AR-8A )  62 

M088100( AR-X  )  558 
M08S057  GS-8A)  170 
M088059( GS-X  )  511 
MC8S065  WK-8A)  65 
M088070 ( WK-X  )  599 

7  M088094 ( AR-8A )  61 

M038101 ( AR-X )  570 
M0880651 WK-8A )  79 
M088U71 ( WK-X )  557 

8  M088094 ( AR-8A )  63 

M088102( AR-X )  572 
M08S065 ( WK-8A )  79 
M088072 ( WK-X  )  555 

9  M088094 ( AR-8A )  65 

M088103( AR-X )  684 
M088065 ( WK-8A )  81 
M088073( WK-X)  571 

10  M088094 ( AR-8A )  65 

M088104( AR-X)  692 
M088065 ( WK-8A  )  81 
M088074 ( WK-X )  572 


Data  Analysis 

The  first  stage  of  data  analysis  was  to  analyze  the  items  using  both  classical 
and  item  response  theory  ( I R T )  methods.  For  each  item  in  a  subtest,  two  classical 
item  indices  were  computed.  One  index  was  p-value,  which  is  an  indication  of  how 
difficult  the  item  is  and  is  defined  as  the  proportion  of  people  answering  the  item 
correctly.  The  other  was  discrimination,  which  is  the  biserial  correlation  between 
item  and  test  scores  and  is  an  indication  of  how  well  the  item  discriminates 
between  low  and  high  ability  examinees. 
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Table  3.  (Concluded) 


11 

M088094 ( 

AR-8A ) 

78 

M088105( 

AR-X  ) 

708 

M088057 

GS-8A ) 

195 

M088060 

GS-X  ) 

587 

12 

M088094 ( 

AR-8A ' 

87 

M088106 

AR-X  ) 

795 

MO88065 ( 

WK-8A ) 

109 

M088075 ( 

WK-X) 

772 

13 

M088094 ( 

'  AR-8A) 

71 

M088107( 

AR-X) 

638 

M088057 ( 

GS-8A ) 

176 

M08806 1 ( 

;  gs-x  ) 

331 

14 

M088094 1 

[  AR-8A  ) 

73 

M088108I 

AR-X  ) 

681 

M088057I 

GS-8A  ) 

189 

M088062I 

'  GS-X  ) 

562 

15 

M088094 ( AR  -8A ) 

69 

M088109 I 

AR-X  ) 

629 

M083057 I 

GS-8A  ) 

174 

M088063 i 

GS-X  ) 

527 

M088065I 

WK-8A  ) 

89 

M088076 I 

[WK-X ) 

612 

16 

M088094 i 

(  AR-8A  ) 

69 

M0881 101 

AR-X  ) 

625 

M088C65 i 

WK-8A) 

86 

M088077 1 

[WK-X ) 

608 

17 

M088094 i 

[  AR-8A ) 

65 

MOSSlili 

AR-X) 

586 

M088065 

WK-8A) 

81 

M088078 

(WK-X) 

568 

18 

M068094 

i  AR-8A ) 

72 

M088112 

AR-X) 

664 

M088057 

(  GS-8A ) 

182 

M088064 

(GS-X ) 

553 

M088065 

(  WK-8A ) 

91 

M088079 ( WK- X ) 

648 

Item  response  theory  indices  were  estimated  using  ASCAL  a  MicroCAT 
testing  system  computer  program  that  estimates  parameters  according  tc  the  three 
parameter  logistic  model  (Office  of  Naval  Research,  1985).  (For  the  interested 
reader,  a  detailed  description  of  item  response  theory  models  can  be  obtained  in 
Lord,  19S0.)  Item  indices  that  were  estimated  included  discrimination  (a 
parameter),  an  indication  of  how  well  the  item  discriminates  between  low  and  high 
ability  examinees;  difficulty  (b  parameter),  an  indication  of  the  item  difficulty;  and 
pseudo-guessing  (c  parameter),  the  probability  of  answering  the  item  correctly  by 
chance. 


For  both  the  classical  and  item  response  theory  analyses,  the  8a  items  were 
analyzed  both  with  and  without  the  additional  items  (i.e.,  overlength  format)  so  that 
test  length  effects  on  estimation  were  consistent.  These  analyses  of  the  items 
from  8a  only  resulted  in  item  statistics  to  be  used  in  the  matching  and  selection  of 
experimental  items. 

The  next  stage  of  data  analysis  was  to  determine  if  items  were  within  a 
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particular  specified  difficulty  range.  Appendix  D  presents  the  targeted  distributions 
of  difficulty  value  ranges  for  each  of  the  power  subtests.  The  distribution  of  both 
classical  item  statistics  and  the  three  item  response  theory  parameter  estimates 
were  calculated.  Examination  of  these  distributions  determined  which  items  met 
minimum  statistical  requirements.  Items  were  deemed  unacceptable  statistically  if 
the  difficulty  or  discrimination  values  were  below  .30.  In  addition,  items  with 
discrimination  values  that  were  negative  and  items  with  responses  to  a  distractor 
that  positively  correlated  with  total  scores  were  deemed  unacceptable. 

Examining  the  statistically  acceptable  experimental  subtest  items  to  see  if  a 
match  could  be  made  with  corresponding  8a  subtest  items  was  performed  to 
determine  if  there  were  sufficient  numbers  of  experimental  items  to  create  six  new 
forms  of  the  ASVAB  in  a  future  effort.  For  example,  the  p-value  and  biserial 
correlation  of  each  experimental  GS  was  compared  to  the  p-value  and  biserial 
correlation  of  each  8a  GS  item  to  determine  which  items  matched.  For  the  AFQT 
subtests,  each  8a  item  must  have  at  least  6  experimental  item  matches.  For  the 
non-AFQT  subtests,  each  8a  item  must  have  at  least  3  experimental  item  matches. 

After  the  matching  process  was  accomplished,  experimental  items  that  were 
statistically  acceptable  and  that  could  be  matched  with  8a  items  were  recommended 
for  use  in  Phase  II  development  of  overlength  forms. 


III.  RESULTS  AND  DISCUSSION 


The  classical  and  IRT  statistics  are  presented  in  Appendix  E  for  the  2754 
experimenta'  and  200  Form  8a  items.  Examination  of  the  distribution  of 
experimental  and  8a  item  statistics  provided  a  means  for  identifying  those 
experimental  items  that  did  not  meet  minimum  statistical  standards  and  provided  a 
comparison  of  the  experimental  and  8a  statistic  distributions. 

Generally,  the  p-values  for  all  experimental  and  8a  subtests  appear  to  be 
distributed  as  expected  and  approximate  the  target  distributions  (Appendix  D).  The 
experimental  p-values  range  from  .000  to  1.000  with  a  majority  of  the  items 
possessing  values  between  .301  and  1.000.  The  8a  difficulty  values  range  from  .301 
to  1.000  with  the  greatest  number  of  values  to  be  matched  above  .500.  Thus,  the 
ability  to  match  experimental  item  p-values  with  8a  item  p-values  was  accomplished. 

The  biserial  correlations  (discrimination  indices)  u'ere  distributed  as  expected 
for  the  experimental  and  8a  subtest.  As  can  be  seen  in  Appendix  E,  the  biserial 
correlations  for  the  experimental  items  range  from  .000  to  1.000  with  a  majority  of 
the  values  from  .201  to  .800.  The  exceptions  to  this  were  GS,  MC,  and  LI  where 
a  significant  number  of  items  possessed  lower  biserial  correlations.  Appendix  E 
also  presents  the  number  of  items  which  possessed  negative  biserial  correlations 
(table  row  labeled  "negative").  These  items  with  negative  biserial  correlations  are 
not  acceptable  for  further  use.  The  8a  biserial  correlations  generally  range  from 
.301  to  1.000  with  the  exception  of  one  item  each  from  AS,  MK,  and  El  with  lower 
biscrials.  Most  of  the  8a  biserial  correlations  were  within  the  .301  and  .700  range 
with  the  exception  of  8  WK  items,  2  PC  items,  5  AS  items,  and  4  MK  items  which 
possess  higher  biserial  correlations.  Again,  the  potential  matching  of  experimental 
biserial  correlations  w'ith  8a  biserial  correlations  W’as  realized. 

To  visually  examine  in  graphic  form  the  simultaneous  matching  of 
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experimental  item  p-values  and  biserial  correlations  with  8a  item  p-values  and 
biserial  correlations,  graphs  were  generated  with  discrimination  on  the  x-axis  and 
difficulty  on  the  y-axis.  Note  that  the  graphs  are  not  presented  in  this  paper; 
however,  an  example  is  presented  in  Figure  1.  Graphing  the  items  in  this  manner 
provided  a  visual  means  of  determining  how  many  experimental  items,  per  subtest, 
did  not  match  an  8a  item  or  fell  below  the  ,30  value  for  p-value  or  biserial 
correlation.  Table  4  presents  in  tabular  form  the  information  contained  in  the 
graphs. 
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Figure  1.  Difficulty  by  Discrimination  for  General  Science:  ForDi  8a 
Booklet  M088057  Versus  Experimental  Booklet  M08S058. 


In  the  simultaneous  matching  process,  relaxed  difference  criteria  were  used. 
A  +.20  point  difference  criterion  for  discrimination  was  chosen  since  biserial 
correlations  have  a  relatively  high  sampling  variance  and  a  tendency  to  flu<  mate 
when  items  are  administered  to  small  samples  or  samples  that  differ  in  ability. 
For  difficulty,  the  difference  criterion  of  +.10  was  chosen.  For  samples  that  more 
(  losely  approximate  the  target  population,  a  more  stringent  criteria  should  be  chosen. 
A  summary  for  each  subtest  of  the  number  of  items  that  were  statistically 
acceptable  and  that  also  matched  8a  items  follows  (see  also  Table  5). 

For  the  GS  subtest  64  experimental  items  possessed  a  difficulty  or 
discrimination  value  below  .30.  These  64  GS  experimental  items  should  not  be 
considered  in  the  development  of  overlength  forms.  The  remaining  159  GS 
experimental  items  appear  to  statistically  match  the  8a  GS  production  items  within 
the  tolerance  limits. 
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Table  4.  Frequency  of  Experimental  and  8a  Items  Crosstabulated 
By  Difficulty  and  Discrimination1* 


General  Science 


Difficulty 

Range 

Unaccept¬ 

able 

.31- 

.40 

Discrimination 
.41-  .51-  .61- 
.50  .60  .70 

ranje 

!80 

.81- 

.90 

.91- 

1.00 

Unacceptable 

20 

9 

3 

4 

1 

- 

- 

- 

.31- 

.40 

8 

8 

4 

(1) 

2 

(3) 

- 

- 

- 

- 

.41- 

.50 

6 

7 

5 

(1  ) 

5 

2 

- 

- 

- 

.51- 

.60 

3 

2 

12 

9 

2 

- 

- 

- 

.61- 

.70 

O 

6 

(1) 

13 

(3) 

5 

(I) 

- 

- 

- 

.71- 

.80 

3 

7 

14 

(3) 

6 

1 

(1) 

- 

- 

- 

.81- 

.90 

3 

5 

(1) 

8 

(3) 

9 

(2) 

2 

1 

- 

- 

.91- 

1.00 

2 

4 

14 

(1) 

3 

(2) 

1 

(2) 

2 

- 

- 

Arithmetic 

Reasoning 

Unacceptable 

10 

2 

1 

1 

- 

- 

- 

.31- 

.40 

3 

2 

6 

(1) 

3 

(1) 

2 

- 

- 

- 

.41- 

.50 

1 

3 

13 

(1) 

14 

(1) 

6 

(2) 

- 

- 

- 

.51- 

.60 

- 

2 

14 

26 

(2) 

14 

(2) 

- 

- 

- 

.61- 

.70 

- 

2 

19 

(1) 

42 

(4) 

19 

2 

- 

- 

.71- 

.80 

- 

6 

17 

4  1 
(4) 

5  1 
(2) 

5 

- 

- 

.81- 

.90 

1 

4 

(1  ) 

23 

(2) 

40 

(1) 

36 

(3) 

12 

- 

- 

.91- 

1 . 00 

2 

8 

(  1  ) 

18 

21 
(  1  ) 

30 

9 

4 

1 

10 


Table  4 ■  (Continued) 


Word  Knowledge 

Diff iculty 
Range 

Unaccept¬ 

able 

.31- 

.40 

Discrimination  ran 
.41-  .51-  .61- 

.50  .60  .70 

«- 

80 

.81- 

.90 

.91- 

1.00 

Unacceptable 

37 

13 

7 

1 

2 

- 

- 

- 

.31- 

.40 

15 

8 

8 

3 

3 

- 

- 

- 

.41- 

.50 

8 

11 

10 

13 

(1) 

4 

- 

- 

- 

.51- 

.60 

6 

13 

15 

(3) 

14 

(1) 

10 

- 

- 

- 

.61- 

.70 

2 

7 

23 

(1) 

24 

(1) 

3 

(1) 

1 

- 

- 

.71- 

.80 

6 

7 

16 

(1) 

23 

(1) 

19 

(1) 

1 

- 

- 

.81- 

.90 

7 

10 

19 

22 

(2) 

20 

(2) 

4 

(1) 

- 

- 

.91- 

1.00 

22 

20 

(1 

40 

) 

60 

(5) 

27 

(6) 

18 

(3) 

11 

(1) 

12 

(3) 

Paragraph 

Comprehension 

Unacceptable 

12 

- 

1 

- 

- 

- 

- 

- 

.31- 

.40 

2 

1 

2 

- 

- 

- 

- 

- 

.41- 

.  50 

1 

- 

1 

2 

- 

- 

- 

- 

.51- 

.60 

- 

4 

5 

(1) 

7 

- 

- 

- 

- 

.61- 

.70 

2 

6 

13 
(1  ) 

9 

(1) 

- 

- 

- 

- 

.71- 

.SO 

1 

3 

17 

(1) 

18 

(1) 

8 

1 

- 

- 

.81- 

.90 

2 

6 

17 

33 

(4) 

12 

(2) 

5 

2 

1 

.91- 

1  .  00 

1 

5 

14 

25 

(1) 

12 

(1) 

14 

(1) 

8 

6 

(1) 

11 


Table  4 ■  (Continued) 


Auto  and  Shop  Information 


Discrimination  range 

Difficulty  Cnaccept-  .31-  .41-  .51-  .61-  .71-  .81-  .91- 

Range _ able _ .40  .50  .60  .70  .80  .90  1.00 


Unacceptable 

16 

4 

3 

1 

1 

“ 

“ 

- 

.31- 

.40 

7 

3 

6 

(1) 

2 

1 

(1) 

- 

- 

.41- 

.50 

2 

3 

(1) 

3 

5 

4 

1 

- 

- 

.51- 

.60 

5 

7 

(1) 

6 

5 

9 

(1) 

5 

(2) 

- 

- 

.61- 

.  70 

1 

7 

(3) 

6 

(2) 

5 

(1) 

9 

(2) 

4 

- 

- 

.71- 

.80 

- 

3 

(1) 

11 

O  CM 

12 

(1) 

4 

(2) 

- 

- 

.81- 

.90 

2 

(1) 

1 

9 

(1) 

11 

(1) 

4 

2 

- 

.91- 

1.00 

1 

1 

5 

(1) 

7 

6 

3 

“ 

- 

Mathematics  Knowledge 

Unacceptable 

13 

6 

11 

4 

6 

1 

- 

- 

.31- 

.40 

3 

4 

12 

10 

8 

(1) 

5 

- 

- 

.41- 

.50 

3 

8 

11 

(1) 

16 

(1) 

12 

(1) 

5 

(1) 

“ 

- 

.51- 

.60 

2 

8 

16 

30 

(2) 

20 

(1) 

7 

1 

- 

.61- 

.70 

- 

3 

15 

34 

(1) 

23 

(2) 

7 

(1) 

1 

- 

.71- 

.80 

- 

5 

17 

(1) 

28 

(2) 

17 

(2) 

12 

(2) 

3 

- 

.81- 

.90 

(1) 

6 

8 

11 

(1) 

17 

(2) 

3 

- 

- 

.91- 

1.00 

_ 

6 

2 

5 

4 

2 

_ 

_ 

(1)  (1) 
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Table  4.  (Concluded) 


Mechanical 

Comprehension 

Difficulty 

Ran«e 

Unaccept¬ 

able 

.31- 

.40 

Discrimination  ran 
.41-  .51-  .61- 

.50  .60  .70 

9?- 

80 

.81- 

.90 

.91- 

1.00 

Unacceptable 

26 

8 

4 

1 

- 

- 

- 

- 

.31- 

.40 

14 

8 

3 

(1) 

1 

- 

- 

- 

- 

.41- 

.  50 

5 

5 

10 

(2) 

2 

(2) 

1 

- 

- 

- 

.51- 

.60 

5 

5 

11 

9 

(3) 

1 

- 

- 

- 

.61- 

.70 

2 

5 

8 

(2) 

9 

(2) 

5 

(3) 

- 

- 

- 

.71- 

.80 

- 

7 

(1  ) 

5 

(2) 

8 

(1) 

3 

(3) 

2 

- 

.81- 

.  90 

- 

7 

6 

(1) 

15 

(1) 

9 

2 

- 

- 

.  91- 

1.00 

- 

1 

3 

(1  ) 

- 

6 

1 

- 

- 

Electronics  Information 

Unacceptable 

15 

(1) 

2 

8 

2 

3 

- 

- 

- 

.31- 

.40 

7 

5 

4 

3 

(1) 

1 

- 

- 

- 

.41- 

.  50 

5 

9 

7 

(3) 

6 

1 

(1) 

- 

- 

- 

.51- 

.60 

4 

8 

10 

6 

(2) 

2 

- 

- 

- 

.61- 

.70 

O 

1 

13 

(1) 

4 

2 

(1) 

- 

- 

- 

.71- 

.80 

4 

6 

7 

(3) 

6 

(  1  ) 

3 

- 

- 

- 

.81- 

.  90 

- 

4 

9 

(1) 

10 

(2) 

3 

(3) 

- 

- 

- 

.91- 

1 . 00 

1 

** 

3 

2 

2 

“ 

a  Unacceptable  is  defined  as  a  value  below  .30  for  difficulty  and 
below  .30  or  negative  for  discrimination.  F requeue ies  in  parentheses 
refer  to  number  of  8a  items  that  fall  within  the  cell;  frequencies 
not  in  parentheses  refer  to  the  number  of  expe  rimental  items. 
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The  AR  subtests  had  25  experimental  items  with  p-values  or  biserials  below 
the  .30  criterion.  In  addition,  8  experimental  items  appeared  not  to  match  8a  items 
statistically.  Therefore  a  minimum  of  33  experimental  items  should  be  culled  and 
not  used  in  the  development  of  overlength  forms.  Of  all  the  subtests,  AR  had  the 
lowest  percentage  (6%)  of  items  that  needed  to  be  culled  and  had  a  total  of  507 
good  items  for  future  consideration. 

Of  the  WK  experimental  items,  126  have  p-values  or  biserials  below  .30  and 
should  not  be  used  in  the  next  phase  of  test  development.  The  remainder  of  the 
WK  experimental  booklet  items  (N  =  499)  statistically  match  8a  items  and  can  bo¬ 
used  in  future  form  development. 

Twenty-two  PC  experimental  items  possess  p-values  or  biserials  below  .30 
and  3  items  do  not  statistically  match  an  8a  item  within  the  +.20  point  difference. 
Therefore,  25  PC  experimental  booklet  items  should  not  be  used  in  later 
development.  The  remaining  246  PC  items  meet  statistical  acceptability  criteria  and 
match  8a  items. 

For  the  AS  experimental  items,  43  items  do  not  meet  the  .30  biserial 

criterion  and  2  items  do  not  statistically  match  an  8a  item.  In  the  development  of 

overlength  forms,  a  minimum  of  45  AS  experimental  booklet  items  should  not  be 
used;  however,  a  total  of  178  AS  items  can  be  considered  for  future  use. 

The  MK  experimental  items  were  found  to  have  49  items  not  meeting  the  .30 

criterion.  However,  the  remainder  of  the  MK  items  appears  to  simultaneously 
match  an  8a  item’s  p-value  and  biserial  correlation.  Thus,  49  MK  items  should  be 
culled  leaving  402  MK  items  for  consideration  in  the  development  of  ovorleng'.h 
forms. 


Table  5.  Number  of  Experimental  Form  20  to  22  Items  to  be 
Considered  in  Overlength  Development 


Min.  H  Items 
needed 

#  to  be 
considered 

ti  NOT  to 
be  used 

% 

loss 

GS 

75 

159 

64 

29% 

AR 

180 

507 

33 

b% 

WK 

210 

499 

126 

20% 

PC 

90 

254 

25 

9% 

AS 

75 

178 

45 

20% 

MK 

150 

402 

49 

n% 

MC 

75 

158 

65 

29% 

El 

60 

137 

53 

28* 

TOTAL 

915 

2,294 

460 
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Among  the  MC  experimental  items,  65  did  not  meet  the  .30  biserial  criterion. 
The  remainder  of  the  MC  items  (N  =  158)  did  statistically  match  8a  items.  A  total 
of  65  MC  experimental  booklet  items  should  not  be  used  in  the  next  test  development 
stage. 


Fifty-three  El  experimental  items  had  p-values  or  biserials  below  .30. 
However,  the  remainder  of  the  El  items  (N  =  137)  statistically  match  8a  items. 
Therefore,  53  El  experimental  booklet  items  should  not  be  considered  in  the 
development  of  overlength  booklets. 

The  8a  and  experimental  items  IRT  a,  b,  and  c  parameters  were  comparable. 
The  a’s,  or  discrimination  index,  for  both  sets  of  items  generally  range  from  .30! 
to  2.500  with  the  greatest  percentage  of  a’s  ranging  from  .501  to  1.400,  The  b’s. 
or  difficulty  parameter,  for  both  experimental  and  8a  items  generally  range  from  - 
3.000  to  3.000  which  were  the  default  lower  and  upper  limits  of  the  estimation 
program. 


Great  caution  is  required  in  interpreting  b  values  of  absolute  high  values  (  + 
or  -  2.5  or  more)  as  most  samples  including  the  present  one  offer  little  statistical 
information  in  these  regions.  Large  sampling  errors  exist  in  these  ranges  but  they 
represent  the  best  point  estimate  available.  Intervals  of  .2  were  created  to  compute 
the  frequencies  of  the  b’s.  Most  of  these  intervals  were  represented  by  1%  to  4% 
of  the  items.  Exceptions  to  this  representation  occurred  within  the  experimental 
OS,  AS,  MC,  and  El  subtests  in  which  the  2.901  to  3.000  range  was  represented  by 
9 %  of  the  items. 

In  summary,  examination  of  both  classical  and  item  response  theory  statistical 
distributions  indicated  that  sufficient  numbers  of  experimental  items  were 
statistically  acceptable  and  were  statistically  comparable  to  corresponding  8a  items. 
Thus,  the  next  stage  of  form  development  involving  the  construction  of  6  overlength 
forms  that  are  parallel  to  one  another  and  to  ASVAB  Form  8a  could  be 
accomplished  in  a  later  effort. 
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APPENDIX  A:  PREVIOUS  ASVAB  BOOKLETS  USED  IN  THE  DEVELOPMENT 

OF  ASVAB  FORMS  20  to  22 


Table  A-l.  Previous  ASVAB  Booklets  Used  In  the  Development 
of  Form  20  to  22  Subtest.  Items 


Booklet  ti 


Sub  tests 


1304 . 1 

,2b  (ASVAB  "1") 

WK  , 

AR. 

MC, 

AI  , 

SI  , 

KI 

ASVAB 

2 

WK, 

AR. 

MC, 

A I  . 

SI  , 

El 

ASVAB 

3 

WK, 

AR, 

MC, 

AI  , 

ST  , 

El 

ASVAB 

■1 

WK, 

AR, 

MC, 

AI  , 

SI  , 

E I 

ASVAB 

1  la 

GS, 

AR, 

WK, 

PC  , 

AS, 

MK, 

ASVAB 

1  lb 

AR, 

WK, 

PC 

ASVAB 

12b 

AR  , 

WK, 

PC 
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APPENDIX  B:  ASVAB  TAXONOMY 


Table  B- 1 ■  ASVAB  Taxonomy 


Subtest 


Content  domain 


Item  % 


General  Science  ( GS ) 

Arithmetic  Reasoning  ( AR ) 

Word  Knowledge  (WK) 

Paragraph  Comprehension  (PC) 
Auto  and  Shop  Information  (AS) 

Mathematics  Knowledge  (MK) 

Mechanical  Comprehend  1  on  (  MC ) 

Ki  ecti'w..  les  Informal  i  on  (Hi) 


Life  Science  45% 

Physical  Science  15% 

Earth  Science 

Rearrangement  of 
Operations  35% 

Equivalent  Fractions  35% 

Percentages  15% 

Arithmetic  Operations  15% 

"Most  nearly  means..." 

stem  60% 

Complete  Sentence  Stem  40% 

Nouns  35% 

Verbs  30% 

Adjectives  35% 

Literal  Recall  40% 

Paraphrase  40% 

Inferences  20% 

Engines  43% 

Boay/Drlve  Train  43% 

Electronics  14% 

Tools  70% 

Materials  30% 

Common  Fractions  25% 

Geometry  25% 

Exponents/Roots  15% 

Equation  Solving  30% 

Factorials  5% 

Simple  Machines  10% 

Compound  Machines  40% 

Complex  Compound 
Machines  20% 

Mechanical  Concepts  30% 

Theory  20% 

C 1 rcu 1 1 s  10% 

Power  /  El  cc  :  r  i  c  1  t.y  4  0% 

Tool s/Regu i 3tors  30% 


IK 


APPENDIX  C:  ASVAB  FORMS  20  to  22  LACKLAND  BOOKLET  CONFIGURATIONS 


Table  C-l.  ASVAB  FORMS  20  to  22  Lackland  Booklet  Configurations 


Lackland  Booklet  M 

Subtests  included 

M088057 

GS( 8a) , 

MC( 8a) ,  AS( 8a ) .  EI(8a) 

M088058-M088063 

GS,  MC, 

AS,  El 

M088064 

GS,  MC, 

AS 

M088065 

WK(  8a ) , 

PC ( 8a ) ,  MK(8a) 

MO88O60 -M088078 

WK ,  PC , 

MK 

M088079 

WK 

M088094 

AR(8a) 

M088095 -M088  112 

AR 
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APPENDIX  D:  TARGET  ASVAB  ITEM  P-VALUES 


Table  D-l ■  Target  P-values 


Distribution  of  item  difficulties  in  each 
operational  length  subtest 


Difficulty 

WK 

PC 

Aft 

MK 

El 

MC 

GS 

AS 

.95-. 90 

3-4a 

1-2 

2-3 

1-2 

1 

1-2 

1-2 

1-2 

.89-. 80 

6-7 

3-4 

5-6 

3-4 

3-4 

3-4 

3-4 

3-4 

.79-. 70 

6-7 

4-5 

5-6 

4-5 

4-5 

4-5 

4-5 

4-5 

.69- .60 

6-7 

4-5 

5-6 

4-5 

4-5 

4-5 

4-5 

4-5 

.59-. 50 

6-7 

4-5 

5-6 

4-5 

4-5 

4-5 

4-5 

4-5 

.49-. 40 

5-6 

2  3 

3-4 

4-5 

4-5 

4-5 

4-5 

4-5 

.39-. 35 

1-2 

1-2 

1-2 

1-2 

1-2 

1-2 

1-2 

1-2 

a  3-4  means  that 

3  to 

4  items  are 

to  be 

within 

this 

difficulty  range. 
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APPENDIX  E:  SUBTEST  CLASSICAL  AND  IRT  STATISTIC  FREQUENCY  INTERVALS 


Table  E-l.  General  Science  Frequencies  of  Item  Statistics  for 
223  Experimental  ana  25  Form  8a  Items. 


P-value 

Experimental 

8  A 

Range 

N 

* 

Range 

N 

% 

.000 

-  .  100 

2 

0.01 

.000 

-  .  1U0 

0 

0.00 

.101 

-  .200 

17 

0.08 

.  101 

.200 

0 

0.00 

.201 

-  .300 

18 

0.08 

.201 

-  .300 

0 

0.00 

.301 

-  .400 

22 

0.10 

.301 

-  .400 

4 

0.16 

.401 

-  .500 

25 

0.11 

.401 

-  .500 

1 

0.04 

.501 

-  .600 

28 

0. 13 

.  501 

-  .600 

0 

0.00 

.601 

-  .700 

26 

0.12 

.601 

-  .700 

5 

0.20 

.701 

-  .800 

31 

G.  14 

.701 

-  .800 

4 

0 . 16 

.801 

-  .900 

2o 

0.13 

.801 

-  .900 

6 

0.24 

.901 

-  l.OOfi 

26 

D  ■  ■ 

0.12 

■  u  m  m 

.901 

-  1.000 

5 

■  ■  ■ 

0.20 

*■  ■  mm 

Biserlal  Correlation 

Experimental 

8A 

Range 

N 

% 

Range 

N 

* 

Negative 

2 

0.01 

Negative 

0 

0.00 

.000 

-  .100 

3 

0.01 

.000 

-  .  1  00 

0 

0.00 

.101 

-  .200 

13 

0.06 

.101 

-  .200 

0 

0.00 

.201 

-  .300 

29 

0.13 

.201 

-  .300 

0 

0.00 

.301 

-  .400 

48 

0.22 

.301 

-  .400 

2 

0.08 

.401 

-  .500 

73 

0.33 

.401 

-  .500 

12 

0.48 

.501 

-  .600 

43 

0. 19 

.501 

-  .600 

7 

0.28 

.601 

-  .700 

9 

0.04 

.601 

-  .700 

4 

0.16 

.701 

-  .800 

3 

0.01 

.  701 

-  .800 

0 

0.00 

.801 

-  .  900 

0 

0.00 

.801 

-  .  900 

0 

0.00 

.  901 

-  1.000 

0 

0.00 

.901 

-  1.000 

0 

0.00 

B  **  m  m  ™  — 

'  ®  “  **“"■*  " 

m  m  rmm 

■•aasapa 

mum 

■  ■  m  m 
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Table  E- 1 .  (Continued) 


IRT  Discrimination  (a)  Parameter 


Experimental 

Range  N  % 


Negative 

■  ■  m 

0 

■  ■■■ 

0.00 

.000 

0 

0.00 

.001 

- 

.100 

0 

0.00 

.101 

- 

.200 

0 

0.00 

.201 

- 

.300 

0 

0.00 

.301 

- 

.400 

b 

0.02 

.401 

- 

.500 

8 

0.04 

.501 

- 

.600 

30 

0.13 

.601 

- 

.700 

26 

0.12 

.701 

- 

.800 

21 

0.09 

.801 

- 

.900 

18 

0.08 

.901 

- 

1.000 

18 

0.08 

1.001 

- 

1  . 100 

11 

0.05 

1  . 101 

- 

1.200 

8 

0.04 

1 .201 

- 

1.300 

8 

0.04 

1 .301 

- 

1 .400 

11 

0.05 

1.401 

- 

1.500 

9 

0.04 

1.501 

- 

1 .600 

7 

0.03 

1  .601 

- 

1.700 

10 

0.04 

1 .701 

- 

1 .800 

7 

0.03 

1  .801 

- 

1.900 

4 

0.02 

1.901 

- 

2.000 

5 

0.02 

2.001 

- 

2.100 

2 

0.01 

2. 101 

- 

2.200 

4 

0.02 

2.201 

- 

2.300 

4 

0.02 

2.301 

- 

2.400 

3 

0.01 

2.401 

- 

2.500 

4 

0.02 

8A 

Range  N 


Negative 

0 

0.00 

.000 

0 

0.00 

.001 

- 

.100 

0 

0.00 

.101 

- 

.200 

0 

0.00 

.201 

- 

.300 

0 

0.00 

.301 

- 

.400 

0 

0.00 

.401 

- 

.500 

6 

0.24 

.501 

- 

.600 

1 

0 . 04 

.601 

- 

.700 

5 

0.20 

.701 

- 

.800 

0 

0.00 

.801 

- 

.900 

2 

0.08 

.901 

- 

1 . 000 

2 

0-08 

1.001 

- 

1 .100 

0 

0.00 

1 .  101 

- 

1 .200 

1 

0.04 

1.201 

- 

1 .300 

1 

0.04 

1.301 

- 

1.400 

2 

0.08 

1.401 

- 

1  .500 

0 

0.00 

1.501 

- 

1  .600 

0 

0.00 

1.601 

- 

1.700 

0 

0.00 

1.701 

- 

1  .800 

0 

0.00 

1.801 

- 

1.900 

0 

0.00 

1.901 

- 

2.000 

1 

0.04 

2.001 

- 

2.100 

1 

0.04 

2.101 

- 

2.200 

0 

0.00 

2.201 

- 

2.300 

2 

0.08 

2.301 

- 

2.400 

1 

0.04 

2.401 

- 

2.500 

0 

0.00 

Table  E- 1 ■  (Concluded) 


IRT  Difficulty  (b)  Parameter 


Experimental  8A 


Range 

N 

% 

Range 

N 

% 

-3.000 

_ 

-2.801 

7 

0.03 

-3.000 

_ 

-2.801 

0 

0.00 

-2.800 

- 

-2.601 

2 

0.01 

-2.800 

- 

-2.601 

1 

0.04 

-2.600 

- 

-2.401 

1 

0.00 

-2.600 

- 

-2.401 

0 

0.00 

-2.400 

- 

-2.201 

5 

0.02 

-2 .400 

- 

-2.301 

0 

0.00 

-2.200 

- 

-2.001 

5 

0.02 

-2.200 

- 

-2.001 

0 

0.00 

-2.000 

- 

-1 .801 

9 

0.04 

-2.000 

- 

-1.801 

3 

0.12 

-1 .800 

- 

-1 .601 

7 

0.03 

-1 .800 

- 

-1.601 

4 

0.16 

-1 .600 

- 

-1.401 

7 

0.03 

-1 .600 

- 

-1.401 

2 

0.08 

-1.400 

- 

-1 .201 

4 

0.02 

-1.400 

- 

-1.201 

0 

0.00 

-1.200 

- 

-1.001 

10 

0.04 

-1.200 

- 

-1 .001 

0 

0.00 

-1.000 

- 

-.801 

7 

0 . 03 

-1.000 

- 

-  .801 

3 

0.12 

-  .800 

- 

-.601 

11 

0.05 

-  .800 

- 

-  .601 

0 

0.00 

-  .600 

- 

-.401 

10 

0.04 

-  .600 

- 

-  .  401 

1 

0.04 

-  .400 

- 

-  .201 

7 

0.03 

-  .400 

- 

-.201 

2 

0.08 

-  .200 

- 

.000 

10 

0.04 

-  .200 

- 

.  000 

0 

0.00 

.003 

- 

.200 

11 

0.05 

.001 

- 

.200 

2 

0.08 

.201 

- 

.400 

10 

0.04 

.201 

- 

.400 

0 

0.00 

.401 

- 

.600 

11 

0.05 

.401 

- 

.600 

1 

0.04 

.601 

- 

.800 

8 

0.04 

.601 

- 

.800 

1 

0.04 

.801 

- 

1.000 

4 

0.02 

.801 

- 

1.000 

3 

0.12 

1.001 

- 

1.200 

3 

0.01 

1.001 

- 

1 . 200 

1 

0.04 

1 .201 

- 

1.400 

6 

0.03 

1.201 

- 

1 .400 

1 

0.04 

1.401 

- 

1  .600 

7 

0.03 

1.401 

- 

1.600 

0 

0.00 

1.601 

- 

1 .800 

9 

0.04 

1.601 

- 

1.800 

0 

0.00 

1.801 

- 

2.000 

6 

0.03 

1.801 

- 

2.000 

0 

0.00 

2.001 

- 

2.200 

6 

0.03 

2.001 

- 

2.200 

0 

0.00 

2.201 

- 

2.400 

6 

0.03 

2.201 

- 

2.400 

0 

0.00 

2.401 

- 

2.600 

10 

0.04 

2.401 

- 

2.600 

0 

0.00 

2.601 

- 

2.800 

2 

0.01 

2.601 

- 

2.800 

0 

0.00 

2.801 

- 

3.000 

22 

0.10 

2.801 

- 

3.000 

0 

0.00 

IRT  Pseudo-Guessing  (c)  Parameter 


Exper imen 

tal 

8A 

Range 

N 

% 

Range 

N 

% 

I nestimable 

1 

0 

00 

Inestimable 

0 

0 

.00 

.000  - 

.  100 

10 

0 

04 

.000  - 

100 

0 

0 

.00 

.101  - 

.200 

39 

0 

17 

.101  - 

200 

5 

0 

.20 

.201  - 

.300 

111 

0 

50 

.201  • 

300 

12 

0 

.48 

.301  - 

.400 

52 

0 

23 

.301  - 

400 

8 

0 

.32 

.401  - 

.  500 

10 

0 

04 

.401  - 

500 

0 

0 

.00 

Table  E-2 .  Arithmetic  Reasoning  Frequencies  of  Item 

Statistics  for  540  Experimental  and  30  Form  8a 
Items 


P-value 

Experimental 

8A 

Range 

N 

% 

Range 

N 

% 

.000 

.  100 

7 

0.01 

.000 

-  .100 

0 

0.00 

.  101 

-  .200 

3 

0.01 

.  101 

-  .200 

0 

0.00 

.201 

.300 

8 

0.01 

.201 

-  .300 

0 

0.00 

.301 

-  .400 

16 

0.03 

.301 

-  .400 

2 

0.07 

.401 

.500 

37 

0.07 

.401 

-  .500 

4 

0.13 

.501 

-  .600 

56 

0.10 

.  501 

-  .600 

4 

0.13 

.601 

-  .700 

84 

0.16 

.601 

-  .700 

5 

0.17 

.701 

-  .800 

120 

0.22 

.701 

-  .800 

6 

0.20 

.801 

-  .900 

116 

0.21 

.801 

-  .900 

7 

0.23 

.901 

-  1.000 

93 

0.17 

.901 

-  1.000 

2 

0.07 

m  m  m 

■  m  m 

as  ■  m  at 

*  *  * 

m  k  =  s 

Biserial  Correlation 

Experimental 

8A 

Range 

N 

* 

Range 

N 

% 

Negative 

10 

0.02 

Negative 

0 

0.00 

.000 

-  .100 

0 

0.00 

.000  -  . 100 

0 

0.00 

.  101 

-  .200 

4 

0.01 

.101  -  .200 

0 

0.00 

.201 

-  .300 

3 

0.01 

.201  -  .300 

0 

0.00 

.301 

-  .400 

29 

0.05 

.301  -  .400 

2 

0.07 

.401 

-  .500 

114 

0.21 

.401  -  .500 

5 

0. 17 

.501 

-  .600 

188 

0.35 

.501  -  .600 

14 

0.47 

.601 

-  .700 

159 

0.29 

.601  -  .700 

9 

0.30 

.701 

.800 

28 

0.05 

.701  -  .800 

0 

0.00 

.801 

-  .900 

4 

0.01 

.801  -  .900 

0 

0.00 

.  901 

-  1.000 

1 

0.00 

.901  -  1.000 

0 

0.00 

* 

1 

■ 

• 

1 

1 

■ 

1 

1 

8 

1 

1 

1 

■ 

•  BE* 

24 


Table  E-2.  (Continued) 


IRT  Dlacrlminatlon  (a)  Parameter 


Experimental  8A 


Range 

N 

* 

Range 

N 

* 

Negative 

0 

0.00 

Negative 

0 

0.00 

.000 

0 

0.00 

.000 

0 

0.00 

.001 

- 

.  100 

0 

0.00 

.001 

- 

.  100 

0 

0.00 

.101 

- 

.200 

0 

0.00 

.  101 

- 

.200 

0 

0.00 

.201 

- 

.300 

0 

0.00 

.201 

- 

.300 

0 

0.00 

.301 

- 

.400 

0 

0.00 

.301 

- 

.400 

0 

0.00 

.401 

- 

.500 

21 

0.04 

.401 

- 

.  500 

1 

0.03 

.501 

- 

.600 

39 

0.07 

.  501 

- 

.600 

2 

0.07 

.601 

- 

.700 

52 

0.10 

.601 

- 

.700 

3 

0.10 

.701 

- 

.800 

65 

0.12 

.701 

- 

.800 

2 

0.07 

.801 

- 

.900 

65 

0.12 

.801 

- 

.900 

4 

0 . 13 

.901 

- 

1.000 

54 

0.10 

.901 

- 

1  .  000 

6 

0.20 

1.001 

- 

1.100 

50 

0.09 

1 . 001 

- 

1  .  100 

1 

0.03 

1  .  101 

- 

1.200 

26 

0.05 

1.101 

- 

1 .200 

2 

0.07 

1.201 

- 

1.300 

18 

0.03 

1 . 201 

- 

1.300 

4 

0.13 

1.301 

- 

1.400 

20 

0 . 04 

1.301 

- 

1.400 

3 

0.10 

1.401 

- 

1.500 

26 

0.05 

1.401 

- 

1.500 

0 

0.00 

1.501 

- 

1  .600 

12 

0.02 

1 . 501 

- 

1.600 

0 

0.00 

1  .601 

- 

1.700 

12 

0.02 

1.601 

- 

1.700 

0 

0.00 

1.701 

- 

1.800 

11 

0.02 

1.701 

- 

1.800 

0 

0.00 

1.801 

- 

1.900 

12 

0.02 

1.801 

- 

1.900 

2 

0.07 

1.901 

- 

2.000 

14 

0.03 

1.901 

2.000 

0 

0.00 

2.001 

- 

2.100 

10 

0.02 

2.001 

- 

2.100 

0 

0.00 

2 .101 

- 

2.200 

15 

0 . 03 

2 .101 

- 

2.200 

0 

0.00 

2.201 

- 

2.300 

10 

0.02 

2.201 

- 

2.300 

0 

0.00 

2.301 

- 

2.400 

4 

0.01 

2.301 

- 

2 .400 

0 

0.00 

2.401 

- 

2.500 

4 

0.01 

2.401 

- 

2.500 

0 

0.00 

Table  E-2.  (Concluded) 


IRT  Difficulty  (b)  Parameter 


Experimental  8A 

Range  N  %  Range  N  % 


-3.000 

i  a  m 

-2.801 

25 

0.05 

-3.000 

a  a 

a  a  a  a  a  be 

-2.801 

a  a  a 

l 

a  a  a  a 

0.03 

-2.800 

- 

-2.601 

17 

0.03 

-2.800 

- 

-2.601 

0 

0.00 

-2.600 

- 

-2.401 

17 

0.03 

-2.600 

- 

-2.401 

1 

0.00 

-2.400 

- 

-2.201 

18 

0.03 

-2.400 

- 

-2.301 

0 

0.00 

-2.200 

- 

-2.001 

14 

0.03 

-2.200 

- 

-2.001 

0 

0.00 

-2.000 

- 

-1.801 

24 

0.04 

-2.000 

- 

-1.801 

2 

0.07 

-1.800 

- 

-1.601 

20 

0.04 

-1.800 

- 

-1.601 

1 

0.03 

-1.600 

- 

-1.401 

30 

0.06 

-1.600 

- 

-1.401 

3 

0.10 

-1.400 

- 

-1 .201 

25 

0.05 

-1 .400 

- 

-1.201 

1 

0.03 

-1.200 

- 

-1.001 

40 

0.07 

-1 .200 

- 

-1.001 

1 

0.03 

-1.000 

- 

-  .801 

27 

0.05 

-1.000 

- 

-.801 

2 

0.07 

-  .800 

- 

-  .601 

36 

0.07 

-  .800 

- 

-.601 

1 

0.03 

-  .600 

- 

-  .401 

38 

0.07 

-  .600 

- 

-.401 

0 

0.00 

-  .400 

- 

-.201 

34 

0.06 

-  .400 

- 

-.201 

3 

0.10 

-  .200 

- 

.000 

28 

0.05 

-  .  200 

- 

.000 

2 

0.07 

.001 

- 

.200 

22 

0.04 

.001 

- 

.200 

3 

0.10 

.201 

- 

.400 

24 

0.04 

.201 

- 

.400 

4 

0. 13 

.401 

— 

.600 

25 

0.05 

.401 

- 

.600 

1 

0.03 

.601 

- 

.800 

16 

0.03 

.601 

- 

.800 

1 

0.03 

.801 

- 

1.000 

18 

0.03 

.801 

- 

1.000 

1 

0.03 

1.001 

- 

1.200 

12 

0.02 

1 .001 

- 

1.200 

1 

0.03 

1.201 

- 

1 .400 

6 

0.01 

1 .201 

- 

1 .400 

1 

0.03 

1.401 

- 

1  .600 

4 

0.01 

1  .401 

- 

1 .600 

0 

0.00 

1.601 

- 

1.800 

3 

0.01 

1.601 

- 

1.800 

0 

0.00 

1.801 

- 

2.000 

2 

0.00 

1.801 

- 

2.000 

0 

0.00 

2.001 

- 

2.200 

0 

0.00 

2.001 

- 

2.200 

0 

0.00 

2.201 

- 

2.400 

0 

0.00 

2.201 

- 

2.400 

0 

0.00 

2.401 

- 

2.600 

2 

0.00 

2.401 

-- 

2.600 

0 

0.00 

2.601 

- 

2.800 

0 

0.00 

2.601 

- 

2.800 

0 

0.00 

2.801 

a  a 

3.000 

13 

mmm 

0.02 

a  a  a  a 

2.801 

■  a  i 

3. COO 

0 

a  a  a 

0.00 

a  a  a  a 

IRT  Pseudo-Guessing  (c)  Parameter 


Experimental 

8A 

Range 

N 

* 

Range 

N 

% 

Inestimable 

0 

0.00 

Inestimable 

0 

0.00 

.000  -  .100 

14 

0.03 

.000  -  .100 

0 

0.00 

.101  -  .200 

81 

0 .15 

.101  -  .200 

10 

0.33 

.201  -  .300 

321 

0.59 

.201  -  .300 

15 

0.50 

.301  -  .400 

100 

0.19 

.301  -  .400 

5 

0.17 

.401  -  .500 

aaaaaaasaaaaa 

24 

0.04 

a  a  a  a 

.401  -  .500 

aaaaaeBCKsaaa 

0 

m  s  a 

0.00 

26 


Table  E-3.  Word  Knowledge  Frequencies  of  Item  Statistics  for 
625  Experimental  and  35  Form  8a  Items 


P-value 

Experimental 

8A 

Range 

N 

* 

Range 

N 

% 

.000 

-  .  100 

4 

0.01 

.000 

-  .100 

0 

0.00 

.  101 

-  .200 

19 

0.03 

.  101 

-  .200 

0 

0.00 

.201 

-  .300 

37 

0.06 

.201 

-  .300 

0 

0.00 

.301 

-  .400 

37 

0.06 

.301 

-  .400 

0 

0.00 

.401 

-  .  500 

46 

0.07 

.401 

-  .  500 

1 

0.03 

.501 

-  .600 

58 

0.09 

.501 

-  .600 

4 

0.11 

.601 

-  .700 

60 

0.10 

.601 

-  .700 

3 

0.09 

.701 

-  .800 

72 

0.12 

.701 

-  .800 

3 

0.09 

.801 

-  .900 

82 

0.13 

.801 

-  .900 

5 

0.14 

.901 

-  1.000 

210 

0.34 

■ikiiH 

.901 

-  1.000 

19 

Etta 

0.54 

mmmm 

Biserial  Correlation 

Experimental 

8A 

Range 

N 

% 

Range 

N 

% 

Negative 

9 

0.01 

Negative 

0 

0.00 

.000 

-  .  100 

10 

0.02 

.000 

-  .  100 

0 

0.00 

.101 

-  .200 

23 

0.04 

.  101 

-  .200 

0 

0.00 

.201 

-  .300 

60 

0 . 10 

.201 

-  .300 

0 

0.00 

.301 

-  .400 

89 

0.14 

.301 

-  .400 

1 

0.03 

.401 

-  .  500 

138 

0.22 

.401 

-  .500 

5 

0.14 

.501 

-  .600 

160 

0.26 

.501 

-  .600 

1  1 

0.31 

.601 

-  .700 

88 

0.14 

.601 

-  .700 

10 

0.29 

.701 

-  .800 

24 

0.04 

.701 

-  .800 

4 

0.11 

.801 

-  .900 

11 

0.02 

.801 

-  .900 

1 

0.03 

.901 

-  1.000 

12 

0.02 

.901 

-  1.000 

3 

0.09 

C  3  o 

■  *  m 

■  ■  SB  M 

■  MEW 

27 


Table  E-3-  (Continued) 

IRT  Discrimination  (a)  Parameter 
Experimental  8A 


Range 

N 

* 

Range 

N 

* 

Negative 

**"o 

oToo 

Negative 

0 

o'oo 

.000 

0 

0.00 

.000 

0 

0.00 

.001 

- 

.  100 

1 

0.00 

.001 

- 

.100 

0 

0.00 

.  101 

- 

.200 

0 

0.00 

.101 

- 

.200 

0 

0.00 

.201 

- 

.300 

0 

0.00 

.201 

- 

.300 

0 

0.00 

.301 

- 

.400 

18 

0.03 

.301 

- 

.400 

1 

0.03 

.401 

- 

.500 

35 

0.06 

.401 

- 

.500 

1 

0.03 

.501 

- 

.600 

40 

0.06 

.501 

- 

.600 

2 

0.06 

.601 

- 

.700 

57 

0.09 

.601 

- 

.700 

8 

0.23 

.701 

- 

.800 

65 

0 . 10 

.701 

- 

.800 

2 

0.06 

.801 

- 

.900 

66 

0.09 

.801 

- 

.900 

4 

0.11 

.901 

- 

1.000 

54 

0.09 

.901 

- 

1.000 

6 

0.17 

1.001 

- 

1 . 100 

46 

0.07 

1.001 

- 

1 . 100 

3 

0.09 

1 . 101 

- 

1.200 

37 

0.06 

1  . 101 

- 

1.200 

1 

0.03 

1.201 

- 

1.300 

36 

0.06 

1.201 

- 

1.300 

1 

0.03 

1.301 

- 

1.400 

27 

0.04 

1.301 

- 

1.400 

1 

0.03 

1.401 

- 

1.500 

19 

0.03 

1.401 

- 

1.500 

0 

0.00 

1.501 

- 

1.600 

17 

0.03 

1.501 

- 

1.600 

0 

0.00 

1.601 

- 

1.700 

24 

0.04 

1.601 

- 

1.700 

0 

0.00 

1.701 

- 

1.800 

18 

0.03 

1.701 

- 

1.800 

0 

0.00 

1.801 

- 

1.900 

17 

0.03 

1.801 

- 

1.900 

0 

0.00 

1.901 

- 

2.000 

21 

0.03 

1 . 901 

- 

2.000 

0 

0.00 

2.001 

- 

2.100 

11 

0.02 

2.001 

- 

2.100 

0 

0.00 

2.101 

- 

2.200 

7 

0.01 

2.101 

- 

2.200 

0 

0.00 

2.201 

- 

2.300 

6 

0.01 

2.201 

- 

2.300 

1 

0.03 

2.301 

- 

2.400 

8 

0.01 

2.301 

- 

2.400 

0 

0.00 

2.401 

- 

2.500 

5 

0.01 

2.401 

2.500 

4 

0.11 

| 


28 


Table  E-3.  (Concluded) 


IRT 

Difficulty 

(b)  Parameter 

Experimental 

Range 

N 

* 

8A 

Range 

N 

% 

a  as  s 

sscaasa 

*  a 

rank 

•  “  * 

-3.000 

- 

-2.801 

79 

0.13 

-3.000 

-  -2.801 

1  1 

0.31 

-2.800 

- 

-2.601 

29 

0.05 

-2.800 

-  -2.601 

3 

0.09 

-2.600 

- 

-2.401 

25 

0.04 

-2.600 

-  -2.401 

2 

0.06 

-2.400 

_ 

-2.201 

27 

0.04 

-2.400 

-  -2.301 

0 

0.00 

-2.200 

— 

-2.001 

28 

0.04 

-2.200 

-  -2.001 

0 

0.00 

-2.000 

- 

-1.801 

18 

0.03 

-2.000 

-  -1.801 

2 

0.06 

-1.800 

- 

-1.601 

25 

0.04 

-1 .800 

-  -1.601 

2 

0.06 

-1.600 

- 

-1.401 

22 

0.04 

-1 .600 

-  -1.401 

0 

0.00 

-1.400 

— 

-1.201 

21 

0.03 

-1 .400 

-  -1.201 

2 

0 . 06 

-1.200 

- 

-1.001 

13 

0.03 

-1 .200 

-  -1.001 

2 

0.06 

-1.000 

- 

-.801 

17 

0.03 

-1.000 

-  -.801 

1 

0.03 

-  .800 

- 

-.601 

22 

0.04 

-  .800 

-  -.601 

1 

0.03 

-  .600 

— 

-.401 

23 

0.04 

-  .600 

-  -.401 

0 

0.00 

-  .400 

_ 

-.201 

22 

0.04 

-  .400 

-  -.201 

0 

0.00 

-  .200 

_ 

.  000 

23 

0.04 

-  .200 

.000 

3 

0.09 

.001 

- 

.200 

24 

0.04 

.001 

.200 

1 

0.03 

.201 

- 

.400 

19 

0.03 

.201 

.400 

0 

0.00 

.401 

- 

.600 

1.8 

0.03 

.401 

.600 

2 

0.06 

.601 

- 

.800 

19 

0.03 

.601 

.800 

0 

0.00 

.801 

- 

1.000 

15 

0.02 

.801 

-  1.000 

3 

0.09 

1.001 

_ 

1.200 

13 

0.02 

1 .001 

-  1.200 

0 

0.00 

1.201 

- 

1.400 

17 

0.03 

1 .201 

-  1.400 

0 

0.00 

1.401 

_ 

1.600 

10 

0.02 

1.401 

-  1.600 

0 

0.00 

1.601 

_ 

1.800 

12 

0.02 

1  .601 

-  1.800 

0 

0.00 

1.801 

- 

2.000 

11 

0.02 

1.801 

-  2.000 

0 

0.00 

2.001 

- 

2.200 

12 

0.02 

2.001 

-  2.200 

0 

n .  oo 

2.201 

- 

2.400 

11 

0.02 

2.201 

-  2.400 

0 

0.00 

2.401 

- 

2.600 

9 

0.01 

2.401 

-  2.600 

0 

0.00 

2.601 

- 

2.800 

5 

0.01 

2.601 

-  2.800 

0 

0.00 

2.801 

- 

3.000 

30 

0.05 

2.801 

-  3.000 

0 

0.00 

1 

* 

1 

R 

H 

M 

n 

rs  m 

BneaBEB 

•  m  m 

•  ■mb 

SBSBBBBI 

a 

1 

1 

ft 

P 

1 

a 

a 

• 

m  *1  m 

m  m  m  m 

IRT  Pseudo-Guessing  (c)  Parameter 


Experimental 

Range  N  % 

ssBaaaasxsK«K  asus 


Inestimable 

2 

0. 

.00 

.000  - 

.  100 

13 

0. 

.  02 

.101  - 

.200 

68 

0. 

,  1 1 

.201  - 

.300 

392 

0 

.63 

.301  - 

.400 

124 

0  . 

,  20 

.401  - 

.  500 

26 

o. 

,04 

■aasBaassasaa  «3*  ass* 


8  A 

Range  N  £ 


Inestimable 

0 

0. 

.00 

.000  - 

.  100 

0 

0. 

.  00 

.101  - 

.200 

1 

0. 

.  03 

.201  - 

.300 

16 

0. 

.46 

.301  - 

.400 

1 5 

0. 

.43 

.401  - 

.500 

3 

0. 

.09 

■  ■■■■■■xBBsaa  m  si  m  mmmm 


29 


Table  E-4 .  Paragraph  Comprehension  Frequencies  of  Item 

Statistics  for  279  Experimental  and  15  Form  8a 
Items 


P-value 

Experimental 

8A 

Range 

N 

% 

Range 

'1 

% 

.000 

-  .  100 

11 

0 .'  04 

.000 

-  .100 

0 

0.00 

.  101 

-  .200 

0 

0.00 

.  101 

-  .200 

0 

0.00 

.201 

-  .300 

2 

0.01 

.201 

-  .300 

0 

0.00 

.301 

-  .400 

5 

0.02 

.301 

-  .400 

0 

0.00 

.401 

-  .500 

4 

0.01 

.401 

-  .  500 

0 

0.00 

.501 

-  .600 

16 

0.06 

.  501 

-  .600 

1 

0.07 

.601 

-  .700 

30 

0.11 

.601 

.700 

2 

0.13 

.701 

-  .800 

48 

0.17 

.701 

-  .800 

2 

0.13 

.801 

-  .900 

78 

0.28 

.801 

-  .  900 

6 

0.40 

.901 

-  1.000 

85 

0.30 

.901 

-  1.000 

4 

0.27 

mm  m 

m  m  «  m 

mam 

SB«a 

Biser ial 

Correlation 

Experimental 

8A 

Range 

N 

* 

Range 

N 

% 

Negative 

6 

0.02 

Negative 

0 

0.00 

.000 

-  .  100 

3 

0.01 

.000  - 

100 

0 

0 . 00 

.101 

-  .200 

4 

0 .01 

.101  - 

200 

0 

0.00 

.201 

-  .300 

8 

0.03 

.201  - 

300 

0 

0 . 00 

.301 

-  .400 

25 

0.09 

.301  - 

400 

0 

0.00 

.401 

-  .  500 

70 

0.25 

.401  - 

500 

3 

0.20 

.501 

-  .600 

94 

0.34 

.501  - 

600 

7 

0.47 

.601 

-  .700 

32 

0.11 

.601  - 

700 

3 

0.20 

.701 

-  .800 

20 

0.07 

.701  - 

800 

1 

0.07 

.801 

-  .900 

10 

0.04 

.801  - 

900 

0 

0.00 

.  901 

-  1.000 

7 

0.03 

m  m  m  m 

.901  -  1 

000 

1 

0.07 

30 


Table  E-4 ■  (Continued) 


IRT 

Discrimination 

(a)  Parameter 

Experimental 

8A 

Range 

N 

% 

Range 

N 

* 

Scac-si 

a  u  = 

escsc 

=  =5  « 

(SEC 

s  a 

OCCBC 

VMM 

mmmtt 

Negative 

0 

0.00 

Negative 

0 

0.00 

.  000 

0 

0.00 

.000 

0 

0.00 

.001 

- 

.100 

0 

0.00 

.001 

- 

.100 

0 

0.00 

.  101 

- 

.200 

0 

0.00 

.101 

- 

.200 

0 

0.00 

.  201 

- 

.300 

0 

0.00 

.201 

- 

.300 

0 

0.00 

.  n  n  i 

- 

.400 

14 

0.05 

.301 

- 

.400 

0 

0.00 

.  5  01 

- 

.  500 

37 

0.13 

.401 

- 

.500 

3 

0.20 

.5'  * 

- 

.600 

37 

0.13 

.  501 

- 

.600 

1 

0.07 

,01)1 

- 

.700 

41 

0. 15 

.601 

- 

.700 

3 

0.20 

.701 

- 

.800 

18 

0 . 06 

.701 

- 

.800 

0 

0.00 

.801 

- 

.900 

12 

0 . 04 

.801 

- 

.900 

1 

0.07 

.101 

- 

1 . 000 

12 

n  .  04 

.  901 

- 

1.000 

1 

0.07 

1.001 

- 

1.100 

8 

o .  03 

1 . 001 

- 

1.100 

* 

X 

0.07 

1 .101 

- 

1 . 200 

2 

0.01 

1.101 

- 

1.200 

0 

0.00 

1.201 

- 

1 . 300 

10 

0.04 

1 . 201 

- 

1.300 

0 

0.00 

1.301 

- 

1.400 

2 

0.01 

1 . 301 

- 

1.400 

0 

0.00 

1.401 

- 

1 . 500 

5 

0.02 

1.401 

- 

1.500 

0 

0.00 

1 . 501 

- 

1 .600 

6 

0 . 02 

1  .  501 

- 

1 .600 

1 

0.07 

1.601 

- 

1 .700 

3 

0.01 

1 .601 

- 

1.700 

0 

0.00 

1 .701 

- 

1.800 

3 

0.01 

1 .701 

- 

1.800 

0 

0.00 

1.801 

- 

1 . 900 

5 

0.02 

1 .801 

- 

1 .900 

0 

o.ot 

1.901 

- 

2 . 000 

7 

0 . 03 

1 . 901 

- 

2.000 

0 

0.00 

2.001 

- 

2.100 

5 

0.02 

2.001 

- 

2.100 

0 

0.00 

2.101 

- 

2.200 

5 

0.02 

2.101 

- 

2.200 

0 

0.00 

2.201 

- 

2.300 

11 

0.04 

2 . 201 

- 

2.300 

0 

0.00 

2.301 

- 

2.400 

18 

0.06 

2.301 

- 

2.400 

0 

0.00 

2.401 

- 

2 . 500 

18 

0 . 06 

2.401 

- 

2.500 

4 

0.27 

*  ■  a 

:  E  m  »  m  m 

s  m  m 

m  m  m 

i  m  m 

S  “  ™  " 

■  MM 

■  ■MM 
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Table  E-4 ■  (Concluded) 


IRT 

Difficulty 

(b)  Parameter 

Experimental 

8A 

Range 

N 

% 

Range 

N 

% 

-3.000 

_ 

-2.801 

34 

0.12 

-3.000 

-2.801 

1 

C  .  07 

-2.800 

- 

-2.601 

12 

0.04 

-2.800 

- 

-2.601 

0 

0.00 

-2.600 

- 

-2.401 

12 

0.04 

-2.600 

- 

-2.401 

0 

0.00 

-2.400 

- 

-2.201 

15 

0.05 

-2.400 

- 

-2.301 

0 

0.00 

-2.200 

- 

-2.001 

16 

0.06 

-2.200 

- 

-2.001 

3 

0.20 

-2.000 

- 

-1.801 

16 

0.06 

-2.000 

- 

-1.801 

2 

0.13 

-1 .800 

- 

-1.601 

18 

0.06 

-1.800 

- 

-1 .601 

1 

0.07 

-1 .600 

- 

-1.401 

19 

0.07 

-1  .600 

- 

-1.401 

0 

0.07 

-1 .400 

- 

-1 .201 

6 

0.02 

-1 .400 

- 

-1 .201 

2 

0.13 

.200 

- 

-1.001 

14 

0.05 

-1.200 

- 

-1.001 

2 

0.13 

-1 .000 

- 

-  .801 

15 

0.05 

-1 . 000 

- 

-  .801 

0 

0.00 

-  .800 

- 

-  .601 

11 

0.04 

-  .800 

- 

-  .60: 

0 

0.00 

-  .600 

- 

-.401 

9 

0  03 

-  .600 

- 

-  .401 

0 

0.00 

-  .400 

- 

-.201 

7 

0.03 

-  .400 

- 

-  .201 

0 

0.00 

-  .  200 

- 

.000 

10 

0.04 

-  .200 

- 

.000 

1 

0.07 

.  001 

- 

.  200 

8 

0.03 

.001 

- 

.200 

1 

0.07 

.201 

- 

.400 

7 

0.03 

.201 

- 

.400 

0 

0.00 

.401 

_ 

.600 

4 

0.01 

.401 

- 

.600 

1 

0 . 07 

.601 

- 

.800 

12 

0.04 

.601 

- 

.800 

0 

0 . 00 

.801 

- 

1.000 

7 

0.03 

.801 

- 

1 . 000 

0 

0.00 

1 .001 

- 

1.200 

2 

0.01 

1  .  001 

- 

1.200 

1 

0.07 

1  .  201 

- 

1 .4  00 

2 

0.01 

1.201 

- 

1.400 

0 

0.03 

1.401 

- 

1.600 

3 

0.01 

1.401 

- 

1  .600 

0 

0.00 

1 .601 

- 

1.800 

3 

0.01 

1.601 

- 

1  .800 

0 

0.00 

1 .801 

- 

2.000 

1 

0.00 

1.801 

- 

2.000 

0 

0.00 

2 . 001 

- 

2.200 

0 

0.00 

2.001 

2.200 

0 

0.00 

2.201 

- 

2.400 

0 

0.00 

2.2C1 

- 

2.400 

0 

0.00 

2 .401 

- 

2.600 

1 

0.00 

2.401 

- 

2.600 

0 

0.00 

2.601 

- 

2.800 

2 

0.01 

2.601 

- 

2.800 

0 

0.00 

2.801 

■  m  t 

3.000 

t 

13 

■  an 

0.05 

n  n  ■  a 

2.801 

■  ■■ 

3.000 

0 

0.00 

M  BM  a 

IRT 

Pseudo-Guessing  (c) 

Parameter 

Experimental 

8A 

Range 

N 

% 

Range 

N 

* 

Inestimable 

1 

0 

.00 

I  nest imable 

0 

.000  - 

.  100 

18 

0 

.06 

.000 

- 

.  100 

1 

.101  - 

.200 

17 

0 

.06 

.101 

- 

.200 

0 

.201  - 

.300  155 

HI 

.  56 

.201 

- 

.300 

8 

0.53 

.301  - 

.400 

57 

0 

.20 

.301 

- 

.400 

3 

0.20 

.401  - 

m  mt 

.  500 

*  *  m  m  m 

31 

m  m 

i 

.  11 

.401 

m  t=  i 

.  500 

3 

0.20 

■  ■  n  ■ 

32 


Table  E-5.  Auto  and  Shop  Information  Frequencies  of  Item 
Statistics  for  223  Experimental  and  25  Form  8a 
Items 


P-value 


Exper 

[mental 

8A 

Range 

N 

% 

Range 

hi 

% 

.000 

100 

7 

0 

03 

.000 

_ 

100 

0 

0 

00 

.101 

- 

200 

8 

0 

04 

.  101 

- 

.200 

0 

0 

00 

.201 

- 

300 

10 

0 

04 

.201 

- 

300 

0 

0 

00 

.301 

- 

400 

19 

0 

09 

.301 

- 

400 

2 

0 

08 

.401 

- 

500 

18 

0 

08 

.401 

- 

500 

1 

0 

04 

.501 

- 

600 

37 

0 

17 

.501 

- 

600 

4 

0 

16 

.601 

- 

700 

33 

0 

15 

.601 

- 

700 

8 

0 

32 

.701 

- 

800 

42 

0 

19 

.701 

- 

.800 

6 

0 

24 

.801 

- 

900 

26 

0 

12 

.801 

- 

900 

3 

0 

12 

.901 

-  1 

BBSS 

000 

B  B  K  B 

23 

0 

10 

■  ■■ 

.  901 

-  1 

BBIkl 

.000 

1 

0 

04 

Biserlal 

Correlation 

Exper] 

[mental 

8A 

Range 

N 

* 

Range 

N 

* 

Negative 

10 

0. 

04 

Negative 

0 

0. 

,00 

.000 

- 

.  100 

4 

0. 

02 

.000 

- 

.100 

0 

0. 

00 

.101 

- 

.200 

8 

0. 

04 

.101 

- 

.  200 

0 

0. 

,00 

.201 

- 

.300 

15 

0. 

07 

.201 

- 

.  300 

1 

0  . 

,04 

.301 

- 

.400 

23 

0. 

10 

.301 

- 

.400 

6 

0, 

,24 

.401 

- 

.500 

53 

0. 

24 

.401 

- 

.500 

5 

0. 

.20 

.501 

- 

.600 

44 

0. 

20 

.501 

- 

.600 

4 

0  . 

,  16 

.601 

- 

.700 

46 

0. 

21 

.601 

- 

.700 

4 

0. 

,  16 

.701 

- 

.800 

20 

0. 

09 

.701 

- 

.800 

5 

0. 

.20 

.801 

- 

.900 

0 

0. 

00 

.801 

- 

.  900 

0 

0. 

.00 

.901 

-  1 

■  b  m  m  i 

.  000 

■  m  m  m 

0 

m  m  m 

0. 

B  ■ 

00 

.901 

-  1 

.  000 

0 

BBS 

0, 

.00 

33 


Table  E-5.  (Continued) 


IRT  Discrimination  (a)  Parameter 


Experimental 

8  k 

Range 

N 

* 

Range 

N 

% 

Negative 

0 

0.00 

Negative 

0 

0.00 

.000 

0 

0.00 

.000 

0 

0.00 

.001 

- 

.  100 

0 

0.00 

.001 

- 

.  100 

0 

0.00 

.  101 

- 

.200 

0 

0.00 

.101 

- 

.200 

0 

0.00 

.201 

- 

.300 

0 

0.00 

.201 

- 

.300 

0 

0.00 

.301 

- 

.400 

4 

0.02 

.  301 

- 

.400 

2 

0.08 

.401 

- 

.500 

8 

0 . 04 

.401 

- 

.500 

4 

0.16 

.501 

- 

.600 

18 

0.08 

.  501 

- 

.600 

1 

0.04 

.601 

- 

.700 

23 

0.10 

.601 

- 

.700 

3 

0.12 

.701 

- 

.800 

9 

0.04 

.701 

- 

.800 

5 

0.20 

.801 

- 

.900 

15 

0.07 

.801 

- 

.900 

0 

0.00 

.901 

- 

1.000 

19 

0.09 

.  901 

- 

.000 

0 

0.00 

1.001 

- 

1.100 

13 

0.06 

1 . 001 

-  1.100 

1 

0.04 

1  . 101 

- 

1.200 

14 

0.06 

1.101 

-  1.200 

1 

0.04 

1.201 

- 

1.300 

8 

0.04 

1.201 

-  1.300 

0 

0.00 

1.301 

- 

1 .400 

13 

0 . 06 

1.301 

-  1.400 

1 

0.04 

1.401 

- 

1.500 

12 

0.05 

1  .401 

-  1.500 

0 

0.00 

1.501 

- 

1 .600 

10 

0.04 

1  .  501 

- 

.600 

1 

0.04 

1  .601 

- 

1  .700 

10 

0 . 04 

1.601 

-  1.700 

0 

0.00 

1.701 

- 

1.800 

9 

0.04 

1 .701 

-  1.800 

1 

0.04 

1  .801 

- 

1.900 

10 

0.04 

1  .SOI 

- 

.900 

0 

0.00 

1  .901 

- 

2.000 

8 

0.04 

1 .901 

- 

2.000 

0 

0.00 

2.001 

- 

2.100 

5 

0.02 

2.001 

- 

2.100 

1 

0.04 

2.101 

- 

2.200 

4 

0.02 

2.101 

- 

2.200 

0 

0,00 

2.201 

- 

2. 30Q 

6 

0.03 

2.201 

-  2.300 

1 

0.04 

2.301 

- 

2.400 

3 

0.01 

2.301 

— 

2.400 

2 

0.08 

2.401 

- 

2.500 

2 

0.01 

2.401 

— 

2.500 

1 

0.04 

m  m  M 

MBS 

■  ■  *  ■ 

a  B 

m  m  m 

34 


Table  E-5 .  (Concluded) 


IRT  Difficulty  (b)  Parameter 


Experimental 

Range 

N 

% 

OSES 

-3.000 

_ 

-2.801 

5 

0.02 

-2.800 

- 

-2.601 

2 

0.01 

-2.600 

- 

-2.401 

2 

0.01 

•2.400 

- 

-2.201 

3 

0.01 

-2.200 

- 

-2.001 

6 

0.03 

-2.000 

- 

-1.801 

7 

0.03 

-1.800 

- 

-1.601 

4 

0.02 

-1  .  oOO 

- 

-1.401 

6 

0.03 

-1  .400 

- 

-1.201 

10 

0.04 

-1  .200 

- 

-1.001 

8 

0.04 

-1.000 

- 

-.801 

8 

0.04 

-  .800 

- 

-.601 

19 

0.09 

-  .600 

- 

-.401 

10 

0.04 

-  .400 

- 

-.201 

10 

0.04 

-  .200 

- 

.000 

14 

0.06 

.001 

- 

.200 

13 

0.06 

.201 

- 

.400 

11 

0 . 05 

.401 

- 

.600 

1 1 

0.05 

.601 

- 

.800 

7 

0.03 

.801 

- 

1 . 000 

10 

0.04 

1  .  001 

- 

1.200 

8 

0 . 04 

1.201 

- 

1.400 

5 

0.02 

1  .401 

- 

1 .600 

7 

0.03 

1 .601 

- 

1.800 

2 

0.01 

1  .80) 

- 

2.000 

4 

0.02 

2.001 

- 

2.200 

3 

0.C1 

2.201 

- 

2 . 400 

3 

0.01 

2.401 

- 

2.600 

1 

0.00 

2.601 

- 

2.800 

2 

0.01 

2.801 

- 

3.000 

22 

0.00 

8A 

Range  N  % 


-3.000 

- 

-2.801 

0 

0.00 

-2.800 

- 

-2.601 

0 

0.00 

-2.600 

- 

-2.401 

0 

0.00 

-2.400 

- 

-2.301 

0 

0.00 

-2.200 

- 

-2.001 

1 

0.04 

-2.000 

- 

-1.801 

2 

0.08 

-1.800 

_ 

-1.601 

0 

0.00 

-1.600 

- 

-1.401 

0 

0.00 

-1.400 

- 

-1.201 

3 

0.12 

-1 .200 

- 

-1.001 

1 

0.04 

-1 . 000 

-.801 

0 

0.00 

-  .800 

- 

-.601 

4 

0.16 

-  .600 

- 

-.401 

1 

0.04 

-  .400 

- 

-.201 

3 

0.12 

-  .200 

- 

.000 

2 

0.08 

.  001 

- 

.200 

2 

0.08 

.201 

- 

.400 

0 

6.00 

.401 

- 

.600 

2 

0.08 

.601 

- 

.800 

1 

0.04 

.801 

- 

1.000 

1 

0.04 

1 .001 

- 

1.200 

0 

0.00 

1.201 

- 

1 .400 

1 

0.04 

1.401 

- 

1  .600 

1 

0.04 

1  .601 

- 

1.800 

0 

0.00 

1.801 

- 

2.000 

0 

0.00 

2.001 

- 

2.200 

0 

0.00 

2.201 

- 

2.400 

0 

0.00 

2.401 

- 

2.600 

0 

0.00 

2.601 

- 

2.800 

0 

0.00 

2.801 

- 

3.000 

0 

0.00 

IRT 

Pseudo- Guess lng 

(c)  Parameter 

Exper imental 

8A 

Range 

N 

% 

Range 

N 

* 

Inestimable 

0 

0.00 

Inestimable 

0 

0.00 

.000  -  .100 

14 

0.06 

.000  -  .100 

3 

0.12 

.101  -  .200 

56 

0.25 

.101  -  .200 

8 

0.32 

.201  -  .300 

96 

0.43 

.201  -  .300 

9 

0.36 

.301  -  .400 

43 

0.19 

.301  -  .400 

4 

0.16 

.401  -  .500 

14 

0.06 

.401  -  .500 

1 

0.04 

m  b  a  m 

m  x  a 

"  “  m  m 

35 


Table  E-6.  Mathematics  Knowledge  Frequencies  of  Item 

Statistics  for  451  Experimental  and  25  Form  8a 
Items 


P-value 


Experimental  8A 

Range  N  %  Range  N  < 


.000 

BBS 

.  100 

2 

m  m  m  m 

0.00 

.000 

.100 

=>  a  b 

0 

0.00 

.101 

- 

.200 

14 

0.03 

.101 

- 

.200 

0 

0.00 

.201 

- 

.300 

25 

0.06 

.201 

- 

.300 

0 

0.00 

.301 

- 

.400 

42 

0.09 

.301 

- 

.400 

1 

0.04 

.401 

- 

.  500 

55 

0.12 

.401 

- 

.500 

4 

0.16 

.501 

- 

.600 

84 

0.19 

.501 

- 

.600 

3 

0.12 

.601 

- 

.700 

83 

0.18 

.601 

- 

.700 

4 

0. 16 

.701 

- 

.800 

S2 

0.18 

.701 

- 

.800 

7 

0.28 

.801 

- 

,900 

45 

0.10 

.801 

- 

.900 

4 

0.16 

.901 

■  b  a 

1.000 

19 

0.04 

.901 

1.000 

2 

SMB 

0.08 

BBSS 

k  m  b 

b  m  m  » 

Biserial 

Correlation 

Experimental 

Range 

N 

% 

8A 

Range 

N 

% 

Negative 

6 

0.01 

Negative 

0 

0.00 

.000  -  .100 

5 

0.01 

.000  -  .100 

0 

0.00 

.101  -  .200 

5 

0.01 

.101  -  .200 

1 

0.04 

.201  -  .300 

5 

0.01 

.201  -  .300 

0 

0.00 

.301  -  .400 

46 

0.10 

.301  -  .400 

1 

0.04 

.401  -  .500 

92 

0.20 

.401  -  .500 

3 

0.12 

.501  -  .600 

138 

0.31 

.501  -  .600 

7 

0.28 

.601  -  .700 

107 

0.24 

.601  -  .700 

9 

0.36 

.701  -  .800 

42 

0.09 

.701  -  .800 

4 

0.16 

.801  -  .900 

5 

0.01 

.801  -  .900 

0 

0.00 

.901  -  1.000 

0 

■  ain 

0.00 

m  m  m  m 

.901  -  1.000 

0 

■  MB 

0.00 

m  a  m  m 

36 


Table  E-6 .  (Continued) 


IRT  Discrimination  (a)  Parameter 


Experimental  8A 


Range 

N 

* 

Range 

N 

* 

Negative 

0 

0.00 

Negative 

0 

0.00 

.000 

0 

0.00 

.000 

0 

0.00 

.001 

- 

.100 

0 

0.00 

.001 

- 

.100 

0 

0.00 

.101 

- 

.200 

0 

0.00 

.101 

- 

.200 

0 

0.00 

.201 

- 

.300 

0 

0.00 

.201 

- 

.300 

0 

0.00 

.301 

- 

.400 

0 

0.00 

.301 

- 

.400 

1 

0.04 

.401 

- 

.500 

16 

0.04 

.401 

- 

.500 

1 

0.04 

.501 

- 

.600 

26 

0.06 

.501 

- 

.600 

1 

0.04 

.601 

- 

.700 

30 

0.07 

.601 

- 

.  70C 

1 

0.04 

.701 

- 

.800 

35 

0.08 

.701 

- 

.800 

4 

0.16 

.801 

- 

.900 

42 

0.09 

.801 

- 

.900 

3 

0.12 

.901 

- 

1.000 

46 

0.10 

.901 

- 

1.000 

2 

0.08 

1.001 

- 

1 .  100 

29 

0.06 

1.001 

- 

1.100 

4 

0.16 

1.101 

- 

1.200 

33 

0.07 

1 . 101 

- 

1.200 

1 

0.04 

1.201 

- 

1 .300 

33 

0.07 

1.201 

- 

1.300 

0 

0.00 

1.301 

- 

1 .400 

26 

0.06 

1.301 

- 

1.400 

1 

0.04 

1.401 

- 

1 . 500 

23 

0.05 

1.401 

- 

1.500 

0 

0.00 

1.501 

- 

1.600 

18 

0.04 

1.501 

- 

1.600 

1 

0.04 

1.601 

- 

1.700 

20 

0.04 

1.601 

- 

1.700 

1 

0.04 

1.701 

- 

1 .800 

19 

0.04 

1.701 

- 

1.800 

0 

0.00 

1.801 

- 

1.900 

16 

0.04 

1.801 

- 

1.900 

0 

0.00 

1.90] 

- 

2.000 

11 

0.02 

1.901 

- 

2.000 

2 

0.08 

2.001 

- 

2 . 100 

12 

0.03 

2.001 

- 

2.100 

0 

0.00 

2.101 

- 

2.200 

10 

0.02 

2.101 

- 

2.200 

0 

0.00 

2.201 

- 

2.300 

0 

0.00 

2.201 

- 

2.300 

2 

0.08 

2.301 

- 

2.400 

2 

0.00 

2.301 

- 

2.400 

0 

0.00 

2.401 

- 

2 . 500 

4 

0.01 

2.401 

- 

2.500 

0 

0.00 

Table  E-6 ■  (Concluded) 


IRT 

Difficulty 

(b)  Parameter 

Experimental 

8A 

Range 

N 

% 

Range 

N 

% 

-3.000 

- 

-2.801 

1 

0.00 

-3.000  - 

-2.801 

2 

0.08 

-2.800 

- 

-2.601 

4 

0.01 

-2.800  - 

-2.601 

1 

0.04 

-2.600 

- 

-2.401 

1 

0.00 

-2.600  - 

-2.401 

0 

0.00 

-2.400 

- 

-2.201 

2 

0.00 

-2.400  - 

-2.301 

0 

0.00 

-2.200 

- 

-2.001 

5 

0.01 

-2.200  - 

-2.001 

0 

0.00 

-2 .000 

- 

-1.801 

5 

0.01 

-2.000  - 

-1.801 

0 

0.00 

-1 .800 

- 

-1.601 

5 

0 . 01 

-1.800  - 

-1.601 

0 

0.00 

-1  .600 

- 

-1.401 

10 

0 . 02 

-1.600  - 

-1.401 

1 

0.04 

-1 .400 

- 

-1.201 

21 

0.05 

-1 .400  - 

-1.201 

0 

0.00 

-1.200 

- 

-1.001 

21 

0.05 

-1.200  - 

-1.001 

4 

0. 16 

-1  .000 

- 

-.801 

15 

0.03 

-1 .000  - 

-  .801 

2 

0.08 

-  .300 

- 

-.601 

33 

0.07 

-  .800  - 

-.601 

1 

0.04 

-  .600 

- 

-.401 

33 

0.07 

-  .600  - 

-.401 

5 

0.20 

-  .400 

- 

-.201 

32 

0.07 

-  .400  - 

-  .201 

0 

0.00 

-  .200 

- 

.000 

27 

0 . 06 

-  .200  - 

.000 

0 

0.00 

.001 

- 

.200 

34 

0.08 

.001  - 

.200 

1 

0.04 

.201 

- 

.400 

32 

0.07 

.201  - 

.400 

1 

0.04 

.401 

- 

.600 

34 

0.08 

.401  - 

.600 

3 

0 . 12 

.601 

- 

.800 

28 

0 . 06 

.601  - 

.800 

2 

0.08 

.801 

- 

1.000 

18 

O.04 

.801  - 

1 .000 

1 

0.04 

1  .001 

- 

1 .200 

17 

0 . 04 

1.001  - 

1 .200 

1 

0.04 

1 .201 

- 

1.400 

19 

0.04 

1.201  - 

1.400 

0 

0.00 

1  .401 

1.600 

16 

0.04 

1 .401  - 

1.600 

0 

0.00 

1.601 

- 

1.800 

4 

0.01 

1.601  - 

1 .800 

0 

0.00 

1  .801 

- 

2.000 

5 

0.01 

1.8C1  - 

2.000 

0 

0.00 

2 .001 

- 

2.200 

7 

0.02 

2.001  - 

2.200 

0 

0.00 

2.201 

- 

2.400 

2 

0.00 

2.201  - 

2.400 

0 

0.00 

2 .401 

- 

2.600 

4 

0 .01 

2.401  - 

2.600 

0 

0.00 

2 .601 

- 

2.800 

0 

0.00 

2.601  - 

2.800 

0 

0.00 

2 .801 

■  tm  l 

3.000 

■ 

16 

b  a  a 

0.04 

2.801  - 

3.000 

0 

0.00 

a  a  a  a 

IRT 

Pseudo-Guessing  (c)  Parameter 

Experimental 

8A 

Range 

N 

* 

Range 

N 

% 

Inestimable 

0 

0 

.00 

Inestimable 

0 

0.00 

.000  - 

.100 

32 

0 

.07 

.000  - 

.100 

3 

0.12 

.101  - 

.200  155 

0 

.34 

.101  - 

.200 

1 1 

0.44 

.201  - 

.300  191 

0 

.42 

.201  - 

.300 

9 

0.36 

.301  - 

.400 

64 

0 

.14 

.301  - 

.400 

2 

0.08 

.401  - 

.500 

9 

0 

.02 

.401  - 

.500 

0 

0.00 

a  m 

b««ii  a 

a  a 

a 

aaoBssc 

=  e  k  a  a  * 

a  =  * 

a  a  a  a 

38 


Table  E-7.  Mechanical  Comprehension  Frequencies  of  Item 

Statistics  for  223  Experimental  and  25  Form  8a 
Items 


P-value 


Experimental 

Range 

* 

8A 

Range 

N 

* 

.000 

-  .  100 

3 

0.01 

.000 

-  .100 

0 

0.00 

.  101 

-  .200 

14 

0.06 

.101 

-  .200 

0 

0.00 

.201 

-  .300 

22 

0.10 

.201 

-  .300 

0 

0.00 

.301 

-  .400 

26 

0.12 

.301 

-  .400 

1 

0.04 

.401 

-  .500 

23 

0.10 

.401 

-  .500 

4 

0.16 

.501 

-  .600 

31 

0.14 

.501 

-  .600 

3 

0.12 

.601 

-  .700 

29 

0.13 

.601 

-  .700 

7 

0.2S 

.701 

-  .800 

25 

0.11 

.701 

-  .800 

7 

0.28 

.801 

-  .900 

39 

0.17 

.801 

-  .900 

2 

0.08 

.901 

-  1.000 

11 

0.05 

.901 

-  1.000 

1 

0.04 

Biserial  Correlation 


Experimental 

Range  N  % 


Negative 

4 

mm  m  m 

0.02 

.000  - 

.100 

5 

0.02 

.  101  - 

.200 

11 

0.05 

.201  - 

.300 

32 

0.14 

.301  - 

.400 

46 

0.21 

.401  - 

.500 

50 

0.22 

.501  - 

.600 

45 

0.20 

.601  - 

.700 

25 

0.11 

.701  - 

.800 

5 

0.02 

.801  - 

.900 

0 

0.00 

.901  - 

1.000 

0 

0.00 

8A 

Range  N  % 


Negative 

0 

0.00 

.000  - 

.100 

0 

0.00 

.101  - 

.200 

0 

0.00 

.201  - 

.300 

0 

0.00 

.301  - 

.400 

1 

0.04 

.401  - 

.500 

9 

0.36 

.501  - 

.600 

9 

0.36 

.601  - 

.700 

6 

0.24 

.701  - 

.800 

0 

0.00 

.801  - 

.900 

0 

0.00 

.901  - 

1.000 

0 

0.00 

39 


Table  E-7.  (Continued) 


IRT 

Discrimination 

(a)  Parameter 

Experimental 

8A 

Range 

N 

* 

Range 

N 

% 

Negative 

0 

0.00 

Negative 

0 

0.00 

.000 

0 

0.00 

.000 

0 

0.00 

.001 

- 

.  100 

0 

0.00 

.001 

- 

.  100 

0 

0.00 

.  101 

- 

.^00 

0 

0.00 

.  101 

- 

.200 

0 

0.00 

.201 

- 

.300 

0 

0.00 

.201 

- 

.300 

0 

0.00 

.301 

- 

.400 

6 

0.03 

.301 

- 

.400 

1 

0.04 

.401 

- 

.500 

12 

0.05 

.  401 

- 

.500 

2 

0.08 

.501 

- 

.600 

18 

0.08 

.501 

- 

.600 

3 

0.12 

.601 

- 

.700 

19 

0.09 

.601 

- 

.700 

3 

0.12 

.701 

- 

.800 

20 

0.09 

.701 

- 

.800 

4 

0.16 

.801 

- 

.900 

13 

0.06 

.801 

- 

.900 

0 

0.00 

.901 

- 

1.000 

11 

0.05 

.901 

- 

1.000 

3 

0.12 

1.001 

- 

1 .100 

19 

0.09 

1.001 

- 

1 . 100 

2 

0.08 

1  . 101 

- 

1.200 

11 

0.05 

1 . 101 

- 

1 .200 

2 

0.08 

1.201 

- 

1.300 

9 

0.04 

1 .201 

- 

1.300 

3 

0.12 

1.301 

- 

1.400 

12 

0.05 

1 .301 

- 

1.400 

1 

0.04 

1.401 

- 

1.500 

10 

0.04 

1.401 

- 

1.500 

0 

0.00 

1.501 

- 

1.600 

15 

0.07 

1.501 

- 

1  .600 

0 

0.00 

1.601 

- 

1.700 

10 

0.04 

1.601 

- 

1.700 

0 

0.00 

1.701 

- 

1.800 

7 

0.03 

1.701 

- 

1.800 

0 

0.00 

1.801 

- 

1.900 

11 

0.05 

1  .801 

- 

1.900 

0 

0.00 

1.901 

- 

2.000 

7 

0.03 

1.901 

- 

2.000 

0 

0.00 

2.001 

- 

2.100 

2 

0.01 

2.001 

- 

2.100 

0 

0.00 

2.101 

- 

2.200 

2 

0.01 

2.101 

- 

2.200 

0 

0.00 

2.201 

- 

2.300 

3 

0.01 

2.201 

- 

2.300 

1 

0.04 

2.301 

- 

2.400 

1 

0.00 

2.301 

- 

2.400 

0 

0.00 

2.401 

- 

2.500 

5 

0.02 

2.401 

- 

2.500 

0 

0.00 

■  m  m 

a  a  a  a 

t  *  K 

=  c  c  a  k 

a  a  a 

a  a  a  a 
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Table  E-7.  (Concluded) 


IRT 

Difficulty 

(b)  Parameter 

Experimental 

8k 

Range 

N 

% 

Range 

N 

% 

-3.000 

-2.801 

2 

0.01 

-3.000 

_ 

-2.801 

0 

0.00 

-2.800 

- 

-2.601 

1 

0.00 

-2.800 

- 

-2.601 

1 

0.04 

-2.600 

- 

-2.401 

1 

0.00 

-2.600 

- 

-2.401 

0 

0.00 

-2.400 

- 

-2.201 

5 

0.02 

-2.400 

- 

-2.301 

1 

0.04 

-2.200 

- 

-2.001 

4 

0.02 

-2.200 

- 

-2.001 

0 

0.00 

-2.000 

- 

-1.801 

5 

0.02 

-2.000 

- 

-1.801 

0 

0.00 

-1.800 

- 

-1.601 

5 

0.02 

-1.800 

- 

-1.601 

1 

0.04 

-1.600 

- 

-1.401 

12 

0.05 

-3  .600 

- 

-1.40, 

1 

0.04 

-1.400 

- 

-1.201 

7 

0.03 

-1 .400 

- 

-1.201 

0 

0.00 

-1.200 

- 

-1.001 

8 

0.04 

-1.200 

- 

-1.001 

1 

0.04 

-1.000 

- 

-.801 

5 

0.02 

-1.000 

- 

-.801 

2 

0.08 

-  .800 

- 

-.601 

11 

0.05 

-  .600 

- 

-.601 

2 

0.08 

-  .600 

- 

-.401 

7 

0.03 

-  .600 

- 

-.401 

1 

0.04 

-  .400 

- 

-  .201 

12 

0.05 

-  .400 

- 

-.201 

5 

0.20 

-  .200 

- 

.000 

15 

0 . 07 

-  .200 

- 

.000 

2 

0.08 

.001 

- 

.200 

8 

0.04 

.001 

- 

.200 

0 

0.00 

.201 

- 

.400 

11 

0.05 

.201 

- 

.400 

2 

0.08 

.401 

- 

.600 

5 

0.02 

.401 

- 

.600 

1 

0.04 

.601 

- 

.800 

3 

0.01 

.601 

- 

.800 

3 

0.04 

.801 

- 

1.000 

11 

0.05 

.801 

- 

1.000 

1 

0.04 

1.001 

- 

1.200 

7 

0.03 

1 .001 

- 

1.200 

3 

0.12 

1.201 

- 

1.400 

10 

0.04 

1.201 

- 

1 .400 

0 

0.00 

1.401 

- 

1  .600 

7 

0.03 

1.401 

- 

1 .600 

0 

0.00 

1.601 

- 

1.800 

8 

0.04 

1 .601 

- 

1.800 

0 

0.00 

1.801 

- 

2.000 

4 

0.02 

1.801 

- 

2.900 

0 

0.00 

2.001 

- 

2.200 

8 

0.04 

2.001 

- 

2.200 

0 

0.00 

2.201 

- 

2.400 

5 

0.02 

2.201 

- 

2.400 

0 

0.00 

2.401 

- 

2.600 

7 

0.03 

2.401 

- 

2.600 

0 

0.00 

2  .  t>01 

- 

2.800 

6 

0.03 

2.601 

- 

2.800 

0 

0.00 

2.801 

■  a 

3.000 

23 

■  ■  m 

0.10 

m  m  m  m 

2.801 

--- 

3.000 

0 

m  m  m 

0.00 

IRT 

Pseudo-Guessing  (c)  Parameter 

Experimental 

8A 

Range 

N 

% 

Range 

N 

* 

Inestimable  0 

0. 

00 

Inestimable 

0 

0.00 

.000  - 

.100  11 

0 

05 

.000 

- 

.100 

2 

0.08 

.101  - 

.200  50 

0 

22 

.  101 

- 

.200 

7 

0.28 

.201  - 

.300  104 

0 

47 

.201 

- 

.300 

14 

0.56 

.301  - 

.400  45 

0 

20 

.301 

- 

.400 

2 

0.08 

.401  - 

-- 

.500  13 

m  m  m  m  m  h 

0 

06 

.401 

... 

.  500 

0 

0.00 

■  m  m  m 

41 


Table  E-8.  Electronics  Information  Frequencies  of  Item 

Statistics  for  190  Experimental  and  20  Form  8a 
Items 


P-value 

Experimental 

8A 

Range 

N 

* 

Range 

N 

% 

.000 

-  .  100 

1 

0.01 

.000 

-  .100 

0 

0.00 

.  101 

-  .200 

10 

0.06 

.101 

-  .200 

0 

0.00 

.201 

-  .300 

19 

0.10 

.201 

-  .300 

1 

0.05 

.301 

-  .400 

20 

0.11 

.301 

-  .400 

1 

0.05 

.401 

-  .500 

28 

0.15 

.401 

-  .500 

4 

0.20 

.601 

-  .600 

30 

0.16 

.501 

-  .600 

2 

0.10 

.601 

-  .700 

22 

0.12 

.601 

-  .700 

2 

0.10 

.701 

-  .800 

26 

0.14 

.701 

-  .800 

4 

0.20 

.801 

-  .  900 

26 

0.14 

.801 

-  .900 

6 

0.30 

.901 

-  1.000 

8 

0.04 

.901 

-  1.000 

0 

■  m  m 

0.00 

•  m  m  m 

Biserial 

Correlation 

Experimental 

8A 

Range 

N 

* 

Range 

N 

% 

Negative 

1 

0.01 

Negative 

0 

0.00 

.000  -  . 100 

8 

0.04 

.000  -  .100 

0 

0.00 

.101  -  .200 

9 

0.05 

.101  -  .200 

0 

0.00 

.201  -  .300 

20 

0.11 

.201  -  .300 

1 

0.05 

.301  -  .400 

36 

0.18 

.301  -  .400 

0 

0.00 

.401  -  .500 

61 

0.32 

.401  -  .500 

8 

0.40 

.501  -  .600 

39 

0.21 

.501  -  .600 

6 

0.30 

.601  -  .700 

17 

0.09 

.601  -  .700 

5 

0.25 

.701  -  .800 

0 

0.00 

.701  -  .800 

0 

0.00 

.801  -  .900 

0 

0.00 

.801  -  .900 

0 

0.00 

.901  -  1.000 

0 

0.00 

.901  -  1.000 

0 

0.00 

42 


Table  E-8 .  (Continued) 


IRT  Discrimination  (a)  Parameter 


Experimental 

Range  N  % 


8k 

Range  N  % 


Negative 

0 

0.00 

Negative 

0 

0.00 

.000 

0 

0.00 

.000 

0 

0.00 

.001 

- 

.100 

0 

0.00 

.001 

- 

.100 

0 

0.00 

.  101 

- 

.200 

0 

0.00 

.101 

- 

.200 

0 

0.00 

.201 

.300 

0 

0.00 

.201 

- 

.300 

0 

0.00 

.301 

- 

.400 

11 

0.06 

.301 

- 

.400 

0 

0.00 

.401 

- 

.  500 

15 

0.08 

.401 

- 

.500 

3 

0.15 

.  501 

- 

.600 

12 

0.06 

.501 

- 

.600 

4 

0.20 

.601 

- 

.700 

20 

0.11 

.601 

- 

.700 

0 

0.00 

.701 

- 

.800 

19 

0.10 

.701 

- 

.800 

1 

0.05 

.801 

- 

.900 

13 

0.07 

.801 

- 

.900 

4 

0.20 

.901 

- 

1  .000 

9 

0.05 

.901 

- 

1 .000 

0 

0.00 

1 . 001 

- 

1  . 100 

8 

0.04 

1.001 

- 

1 .100 

0 

0.00 

1  .  101 

- 

1 .200 

9 

0.05 

1  .  101 

- 

1  .200 

0 

0.00 

1.201 

- 

1  .300 

10 

0.05 

1 .201 

- 

1 .300 

0 

0.00 

1.301 

- 

1 .400 

4 

0.02 

1.301 

- 

1 .400 

1 

0.05 

1.401 

- 

1  .500 

6 

0.03 

1 .401 

- 

1.500 

0 

0.00 

1 . 501 

- 

1  .600 

5 

0.03 

1.501 

- 

1  .600 

1 

0.05 

1.601 

- 

1  .700 

9 

0.05 

1 .601 

- 

1 .700 

0 

0.00 

1.701 

- 

1  .800 

11 

0.06 

1.701 

- 

1 .800 

0 

0.00 

1 .801 

- 

1  .  900 

6 

0.03 

1.801 

- 

1.900 

0 

0.00 

1 . 901 

- 

2.000 

4 

0.02 

1 . 901 

- 

2.000 

0 

0.00 

2.001 

- 

2 . 100 

0 

0.00 

2.001 

- 

2.100 

0 

0.00 

2 .101 

- 

2.200 

4 

0.02 

2.101 

- 

2.200 

0 

0.00 

2.201 

- 

2.300 

4 

0.02 

2.201 

- 

2.300 

2 

0.10 

2.301 

- 

2.400 

5 

0.03 

2.301 

- 

2.400 

o 

0.10 

2.401 

- 

2.500 

6 

0.03 

2.401 

- 

2.500 

2 

0.10 
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Table  E-8.  (Concluded) 


IRT 

Difficulty 

(b)  Parameter 

Experimental 

% 

8A 

% 

Range 

N 

Range 

N 

-3.000 

_ 

-2.801 

1 

0.01 

-3.000 

-2.801 

0 

0.03 

-2.800 

- 

-2.601 

0 

0.01 

-2.800 

- 

-2.601 

0 

0.00 

-2.600 

- 

-2.401 

2 

0.01 

-2.600 

- 

-2.401 

0 

0.00 

-2.400 

- 

-2.201 

3 

0.02 

-2.400 

- 

-2.301 

0 

0.00 

-2.200 

- 

-2.001 

1 

0.01 

-2.200 

- 

-2 . 001 

0 

0.00 

-2.000 

- 

-1.801 

4 

0.02 

-2.000 

- 

-1.801 

1 

0.05 

-1.800 

_ 

-1.601 

3 

0.02 

-1.800 

- 

-1 .601 

0 

0.00 

-1.600 

- 

-1 .401 

8 

0 . 04 

-1.600 

- 

-1.401 

0 

0.00 

-1.400 

- 

-1.201 

6 

0 . 03 

-1.400 

- 

-1.201 

4 

0.20 

-1.200 

- 

-1.001 

12 

0 . 06 

-1.200 

- 

-1 .001 

1 

0.05 

-1.000 

- 

-  .801 

10 

0.05 

-1.000 

- 

-.801 

2 

0.10 

-  .800 

- 

-.601 

6 

0.03 

-  .800 

- 

-.601 

1 

0.05 

-  .600 

- 

-.401 

7 

0.04 

-  .600 

- 

-.401 

0 

0.00 

-  .400 

- 

-.201 

4 

0.02 

-  .400 

- 

-.201 

1 

0.05 

-  .200 

- 

.  000 

11 

0 . 06 

-  .200 

- 

.000 

1 

0.05 

.001 

- 

.200 

4 

0 . 02 

.001 

- 

.200 

1 

0.05 

.201 

- 

.400 

7 

0.04 

.201 

- 

.400 

0 

0.00 

.401 

- 

.600 

12 

0.06 

.401 

- 

.600 

1 

0.05 

.601 

- 

.800 

9 

0.05 

.601 

- 

.800 

1 

0.05 

.801 

- 

1.000 

7 

0.04 

.801 

- 

1  .  000 

2 

0.10 

1.001 

- 

1.200 

5 

0.03 

1.001 

- 

1  .  20C 

2 

0.10 

1.201 

- 

1.400 

7 

0.04 

1.201 

- 

1.400 

0 

0.00 

1.401 

- 

1.600 

14 

0.07 

1.401 

- 

1  .600 

1 

0.05 

1.601 

- 

1.800 

9 

0.05 

1.601 

- 

1.800 

0 

0.00 

1.801 

- 

2.000 

2 

0.01 

1.801 

- 

2.000 

0 

0.00 

2.001 

- 

2.200 

5 

0.03 

2.001 

- 

2.200 

0 

0.00 

2.201 

- 

2.400 

6 

0.03 

2.201 

- 

2.400 

1 

0.05 

2.401 

- 

2.600 

4 

0.02 

2.401 

- 

2.600 

0 

0.00 

2.601 

- 

2.800 

1 

0.01 

2.601 

- 

2.800 

0 

0.00 

2.801 

3.000 

20 

0.11 

2.801 

3.000 

0 

0.00 

TRT 

P seudo -Guess ing 

(c)  Parameter 

Experimental 

% 

8A 

% 

Range 

N 

Range 

N 

B  50 

assn  ■ 

a  m 

a  a 

m 

s  a 

a  sa  e  a  a  a 

SEE 

9  m  a  a 

Inestimable 

0 

0. 

00 

Inestimable 

0 

0.00 

.000  - 

.  100 

11 

0. 

06 

.000 

- 

.100 

2 

0.10 

.101  - 

.200 

40 

0  . 

21 

.101 

- 

.200 

0 

0.00 

.201  - 

.300 

84 

0. 

44 

.201 

- 

.300 

12 

0.60 

.301  - 

.400 

43 

0. 

23 

.301 

- 

.400 

6 

0 . 30 

.401  - 

.  500 

12 

0. 

06 

.401 

- 

.500 

0 

0.00 

1 

« 

V 

1 

1 

1 

1 

"  * 

ssea  ■ 

a  a 

m  m 

1  a  m 

a  a  a  e  a 

=  = 

=  a  n  s  a  e 

a  a  a 

9  ■  u  m 

i 
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